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ABSTRACT 


This  paper  reports  the  results  of  a  study  that  compared  the  effectiveness  of  the 
traditional  school-house  and  an  interactive  multimedia  system  in  training  Repair  Party 
Leaders.  The  study  was  conducted  over  a  period  of  three  months  at  the  Repair  Party 
Leader  School,  Fleet  Training  Center,  San  Diego,  California.  Subjects  participating  in 
the  study  ranged  from  mid-grade  enlisted  sailors  (second  class  petty  officers)  to  junior 
officers  (Ensign  through  Lieutenant  Commander). 

Findings  from  three  separate  analyses  are  presented.  The  data  analysis  provides  a 
basis  for  concluding  that  the  interactive  system  may  be  an  effective  way  to  deliver 
training  to  members  of  the  shipboard  damage  control  organization  in  a  manner  in  keeping 
with  the  “Smart  Ship”  concept.  Exit  level  of  knowledge  comparisons  between  personnel 
that  used  the  IDCTT  (RPL)  as  a  supplement  to  the  standard  RPL  school  curriculum  and 
those  that  did  not  are  examined,  and  final  results  and  recommendations  are  provided. 
Recommendations  for  improvement  of  the  system,  as  well  as  further  research  topics  are 
also  provided. 
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EXECUTIVE  SUMMARY 


Over  the  past  several  years,  computer-based,  interactive  training  has  become  an 
important  part  of  the  military’s  education  system.  This  technology,  once  out  of  reach  due 
to  cost,  is  gaining  acceptance  within  the  Department  of  the  Navy  as  a  means  of  either 
supplementing  or  replacing  school-based  training.  With  the  decreasing  cost  of  Windows- 
based  personal  computers,  the  Navy  is  aggressively  pursuing  development  and 
procurement  of  interactive  courseware  that  will,  by  providing  improved,  ‘just  in  time” 
training,  enhance  the  overall  combat  readiness  of  ships  and  aircraft  within  the  force. 

The  IDCTT  (RPL)  is  a  U.S.  Navy-developed  software  program  that  provides 
realistic  integrated  shipboard  casualty  control  training.  This  system  utilizes  a  large 
database  and  numerous  video  clips  to  provide  the  trainee  with  a  very  realistic  training 
environment.  The  intent  of  IDCTT  (RPL)  is  to  provide  the  student  with  practice  in 
collecting  and  processing  information,  coordinating  responses,  and  maintaining  the 
overall  damage  control  picture.  These  skills  are  difficult  to  develop  outside  of  actually 
imposing  damage  control  scenarios  and  then  evaluating  the  Repair  Party  Leader’s  actions 
and  responses.  IDCTT  provides  the  RPL  candidate  with  unlimited  opportunities  to  “test 
his  or  her  mettle”  against  the  computer,  practicing  personnel  and  equipment  management, 
prioritization  of  effort  and  event  sequencing.  This  software  also  provides  the  support 
necessary  to  conduct  simultaneous  interactive  team  training.  The  ability  to  modify 
scenarios  at  the  local  command  level  enables  the  ship’s  damage  control  training  team  to 
tailor  training  to  specifically  target  areas  of  weakness  identified  during  drills  or  other 
training  sessions.  IDCTT  (RPL)  provides  immediate  student  performance  evaluation  and 
feedback,  reinforcing  the  training  received  by  the  user.  Finally,  cross-platform 
connectivity  between  Windows-,  Unix-  and  OS2-based  operating  systems  allows  for 
system  flexibility. 

Repair  Party  Leaders  are  trained  at  Fleet  Training  Centers  in  San  Diego, 
California,  and  Norfolk,  Virginia.  The  course  of  instruction  emphasizes  advanced 
damage  control  theory  through  classroom  lecture,  and  provides  for  some  practical 
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application  of  topics  presented  in  the  classroom.  However,  the  limited  nature  of  the 
course,  as  well  as  restricted  facilities,  allow  the  student  little  opportunity  to  actually 
coordinate  the  activities  of  a  Repair  Locker,  manage  personnel  and  equipment,  or 
exercise  decision-making  techniques.  It  is  thought  that  use  of  IDCTT  (RPL)  as  an 
educational  augment  at  the  school  will  enhance  this  opportunity,  and  result  in  an 
improved  graduate,  one  more  ready  to  contribute  to  his  or  her  ship’s  damage  control 
readiness. 

The  study  was  conducted  using  students  from  three  RPL  school  classes  that 
convened  in  March,  April  and  May  1997.  For  this  research,  each  student  involved  was 
given  the  option  of  not  participating  in  the  study.  Of  the  73  students  attending  the  school 
during  the  three-month  test  period,  all  opted  to  participate  in  the  experiment.  The  March 
and  April  classes  were  randomly  divided  into  two  groups  (control  and  treatment),  with 
the  treatment  group  having  their  training  augmented  by  use  of  the  IDCTT  (RPL).  In 
April,  however,  the  treatment  group  was  comprised  of  volunteers. 

For  the  study,  the  control  group  proceeded  through  the  school-prescribed  training 
track.  The  treatment  group  also  passed  through  the  normal  curriculum,  but  was  also 
given  standard  scenarios  during  the  course  requiring  the  use  of  the  IDCTT.  During  week 
two  of  the  course,  each  student  sat  before  an  oral  board  (examination).  These  oral  boards 
were  conducted  by  two  retired  senior  Naval  officers,  both  with  command  experience. 
The  “proctors”  had  no  knowledge  of  the  subject’s  training  track,  ensuring  an  unbiased 
evaluation.  Questions  asked  at  this  board  were  designed  to  determine  level  of  knowledge 
based  on  standards  established  by  the  Repair  Party  Leader  Personnel  Qualification 
Standard. 

Data  analysis  revealed  that  the  students  that  used  IDCTT  (RPL)  as  an  augment  to 
the  established  curriculum  performed  significantly  better  on  the  oral  boards  than  did 
those  in  the  standard  training  track.  In  feet,  the  treatment  group  posted  a  mean  oral  board 
score  of  60.78,  compared  with  48.47  for  the  control  group.  These  figures  correspond  to  a 
quantitative  learning  advantage  of  20.26%,  a  value  that  corresponds  with  results  from 
similar  studies  regarding  the  effectiveness  of  interactive  training  systems.  The  May  class 
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volunteers  performed  even  better,  showing  a  quantitative  learning  advantage  of  77%. 

Based  on  the  results,  it  can  be  concluded  that  use  of  IDCTT  (RPL)  at  the  Repair 
Party  Leader  School  in  San  Diego  did  have  a  positive  impact  on  the  oral  board  scores 
recorded  by  the  treatment  group.  This  technology  holds  great  promise  for  improving  the 
training  available  to  shipboard  personnel.  Improved  training  will  contribute  to  enhanced 
damage  control  readiness,  and  will  enable  ships  to  meet  the  requirements  of  total  ship 
survivability. 
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L  INTRODUCTION 


Over  the  past  several  years,  computer-based,  interactive  training  has  become  an 
important  part  of  the  military’s  education  system.  This  technology,  once  out  of  reach  due 
to  cost,  is  gaining  acceptance  within  the  Department  of  the  Navy  as  a  means  of  either 
supplementing  or  replacing  school-based.  With  the  decreasing  cost  of  Windows-based 
personal  computers,  the  Navy  is  aggressively  pursuing  development  and  procurement  of 
interactive  courseware  that  will,  by  providing  improved,  “just  in  time”  training,  enhance 
the  overall  combat  readiness  of  ships  and  aircraft  within  the  force. 

Systems  that  have  been  developed  for  use  by  the  Department  of  the  Navy  include 
the  Interactive  Damage  Control  Training  Technology  (IDCTT)  for  Damage  Control 
Assistants,  as  well  as  interactive  multimedia  courseware  used  to  provide  challenging 
decision-making  scenarios  to  junior  officers  in  command  of  the  CYCLONE-class  PC’s 
(coastal  patrol  boats).  In  the  instance  of  IDCTT,  the  primary  goal  of  the  software  is  to 
provide  a  supplement  to  training  received  in  the  traditional  schoolhouse.  On  the  other 
hand,  the  system  employed  for  the  commanding  officers  of  the  PC’s  is  used  to  expose  the 
individual  to  “the  worst  day  at  sea”,  causing  him  to  have  to  make  quick  and  accurate 
decisions  in  order  to  avert  catastrophe  and  complete  an  assigned  mission.  (J.  Miller, 
personal  communication,  September  26, 1996) 

The  purpose  of  this  thesis  is  to  determine  the  effectiveness  of  the  use  of  an 
interactive  multimedia  system  in  the  training  of  Repair  Party  Leader  (RPL)  candidates  in 
order  to  prepare  them  to  assume  the  responsibilities  associated  with  this  position  in  the 
ship’s  damage  control  organization.  This  research  will  examine  whether  the  use  of 
interactive  courseware  to  supplement  training  received  by  students  at  the  Repair  Party 
Leader  course  conducted  at  U.S.  Navy  Fleet  Training  Centers  will  result  in  a  graduate  that 
is  closer  to  achieving  qualification  as  a  Repair  Party  Leader.  It  is  predicted  that  a  student 
that  has  used  IDCTT  (RPL)  in  conjunction  with  the  Repair  Party  Leader  Course  will  have 
a  significantly  higher  exit  level  of  knowledge  than  one  trained  by  traditional  schoolhouse 
methods  alone. 
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II.  BACKGROUND  OF  INTERACTIVE  LEARNING 


A.  CONCEPTS  OF  INTERACTIVE  MULTIMEDIA  USED  IN  EDUCATION 

AND  TRAINING 

Computer-based  instruction  (CBI)  using  interactive  multimedia  has  gained 
increased  acceptance  over  the  last  20  years  as  an  effective  and  inexpensive  means  of 
providing  training  to  military  personnel.  This  method  is  well  suited  to  military 
applications  as  training  resources  dwindle  and  the  requirement  for  personnel  able  to 
operate  and  maintain  sophisticated  systems  increases.  As  a  computer-based  system, 
IDCTT  (RPL)  serves  as  a  tool  with  which  to  assist  the  Repair  Party  Leader  in  developing 
decision-making  skills  based  on  a  foundation  of  knowledge  in  basic  and  advanced 
damage  control. 

Spiro  (as  cited  in  Seidel  and  Weddle,  1987)  describes  an  “Experience- 
Consolidation  System”  as  a  computer-based  teaching  tool  that  assists  the  user  in 
developing  judgement  and  intuition  that  will  result  in  better  decision  making  skills. 
IDCTT  (RPL)  can  be  categorized  as  such  a  system  in  that  it  focuses  on  the  user’s  ability 
to  make  correct  decisions  and  instigate  action  in  a  time-compressed  situation. 
Experience-Consolidation  Systems  teach  what  is  normally  associated  with  repetition  and 
practice  in  the  field  by  helping  to  establish  patterns  for  problem  solving,  and  causing  the 
user  to  “think  for  himself.”  (Spiro,  1987) 

Taylor  (as  cited  in  Alessi  &  Trollip,  1991)  classified  instructional  computing  into 
three  categories:  tutor,  tool  and  tutee.  The  tool  and  tutee  categories  do  not  apply  to 
IDCTT  (RPL),  and  are  not  discussed  in  this  research.  For  computer-based  instruction 
like  that  presented  by  IDCTT  (RPL),  the  system  serves  primarily  as  a  sort  of  tutor. 
Although  it  does  not  walk  the  trainee  through  a  scenario,  IDCTT  (RPL)  provides 
information  and  assessment  via  end  of  session  feedback.  Additionally,  diming  the 
conduct  of  the  scenario,  the  user  has  access  to  several  options  that  serve  as  memory  aids, 
or  prompts,  that  will  cause  the  user  to  draw  on  previously  obtained  knowledge  of  damage 
control  theory,  procedures  and  doctrine  in  order  to  make  the  correct  decision. 
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A  difficulty  with  any  computer-based  instruction  is  that  for  the  system  to  provide  any 
positive  result,  the  user  must  be  comfortable  with  the  operation  of  a  personal  computer 
(Alessi  and  Trollip,  1991).  In  today’s  military,  it  is  likely  that  more  recently-inducted 
members  will  have  frequently  used  computers  prior  to  joining,  where  as  personnel  that 
have  been  in  the  service  for  several  years  may  never  have  been  exposed  to  them  This 
disparity  may  result  in  differences  in  results  obtained  by  users  of  the  computer-based 
learning  system,  and  may  discourage  those  that  have  little  or  no  computer  literacy. 

In  order  to  maximize  results,  the  computer  should  be  used  in  training  situations 
where  it  is  likely  to  be  beneficial  (Alessi  and  Trollip,  1991).  Providing  training  to 
candidates  or  serving  Repair  Party  Leaders  is  a  good  application  as  the  traditional 
training  cost  is  high  (Le.  the  individual  Sailor  is  absent  from  his  ship  for  12  days),  and  the 
nature  of  the  qualification  process  requires  many  repetitions  of  drills.  IDCTT  (RPL)  can 
provide  low-cost  training  to  individual  students,  reducing  the  impact  this  training  has  on 
the  remainder  of  ship’s  company. 

Alessi  and  Trollip  (1991)  describe  instruction  as  a  four-part  process.  The  initial 
phase  is  that  of  “presenting  the  information.”  In  the  case  of  the  Repair  Party  Leader,  this 
is  accomplished  through  the  trainee’s  completion  of  prerequisite  Personnel  Qualification 
Standards  and  attainment  of  appropriate  qualifications.  Phase  two  of  the  process  is  that 
of  “guiding  the  student.”  IDCTT  (RPL)  provides  a  modicum  of  guidance  via  “pull¬ 
down”  menu  options  that  can  be  selected  by  the  user,  as  well  as  prompts  provided  by  the 
“phone-talker”  and  messages  from  the  On-Scene  Leader  and  Damage  Control  Central 
The  third  and  fourth  segments  of  the  process  involve  “practicing  the  student”  and 
“assessing  learning.”  IDCTT  (RPL)  uses  the  concept  of  drills  to  enhance  student 
performance,  but,  in  the  strictest  sense,  does  not  provide  training  for  the  user.  Alessi  and 
Trollip  (1991,  p.  108)  stress  that  single  drill  sessions  should  not  exceed  15  minutes  in 
order  to  prevent  user  boredom  or  fatigue.  As  currently  configured,  IDCTT  (RPL)  drill 
scenarios  require  between  six  and  15  minutes  to  complete. 

Computer-based  drills  have  the  ability  to  provide  a  flexible,  interesting  learning 
environment  to  the  user.  The  employment  of  sound  and  video  clips,  as  well  as 
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sophisticated  visual  graphics,  provides  the  student  with  a  realistic  presentation  of 
situations  and  any  impact  his  or  her  decisions  may  have  on  the  progress  of  the  scenario. 
This  multimedia  environment  may  stimulate  further  study  into  the  concept  being  pursued, 
and,  as  a  result,  a  higher  level  of  user  ability. 

B.  APPLICATIONS  OF  COMPUTER-BASED,  INTERACTIVE  AND 

MULTIMEDIA  INSTRUCTION 

As  mentioned  earlier,  the  Navy  is  in  the  process  of  fielding  several  computer- 
based  training  systems.  The  U.S.  Army  has  also  invested  resources  in  the  development 
of  interactive  training  systems  in  an  effort  to  more  efficiently  spend  scarce  training 
dollars.  One  such  system  developed  by  the  U.S.  Army  Signal  Center  at  Fort  Gordon, 
Georgia,  involves  the  use  of  interactive  videodisc  technology  to  train  communications- 
electronics  military  occupational  specialties.  This  system  uses  a  microcomputer  and  laser 
videodisc  to  provide  an  interactive  learning  environment  for  soldiers  in  these  fields 
(Winkler  and  Polich,  1990). 

Another  application  of  interactive  training  employed  within  the  Navy  involves  the 
training  of  U.S.  Marines  in  field  first  aid.  (Dr.  Eric  Allely,  personal  communication, 
January  15,  1997)  It  is  desirable  that  when  in  combat,  every  Marine  be  capable  of 
performing  life-saving  measures  on  a  wounded  comrade,  providing  support  until  other 
field  medical  personnel  arrive  to  treat  the  casualty.  A  system  similar  in  design  to  IDCTT 
has  been  developed  for  this  purpose,  and  field-testing  indicates  that  it  is  an  effective  way 
to  train  Marines  in  this  very  valuable  skill. 

C.  PREVIOUS  FINDINGS 

Interactive  learning  attempts  to  reduce  training  time  required  to  raise  an 
individual  to  a  certain  level  of  task  proficiency.  Studies  conducted  by  Orlansky  and 
String  in  1979,  and  Orlansky,  String  and  Chatelier  in  1982  (as  cited  by  Funaro  and 
Lane  in  Seidel  and  Weddle,  1987)  showed  that  in  typical  settings,  computer-based 
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learning  applications  “achieved  an  effectiveness  equal  or  slightly  superior  to  conventional 
methods  with  a  time  savings  on  the  order  of  30  percent.”  While  this  research  does  not 
address  the  issue  of  time,  the  use  of  IDCTT  (RPL)  as  a  means  of  reducing  time  required 
to  earn  qualification  as  a  Repair  Party  Leader  is  an  area  that  provides  opportunity  for 
further  study. 

Unfortunately,  previous  studies  do  not  indicate  that  computer-based  instruction 
provides  a  noticeable  improvement  when  compared  with  training  conducted  using 
conventional  methods,  except  in  the  application  of  reducing  time  required  to  learn  a  task. 
In  feet,  many  studies  indicate  that  there  is  no  significant  difference  between  results 
achieved  by  interactive  and  conventional  instruction.  Clark  and  Craig  (as  cited  in 
McKenna,  1995)  found  that  the  use  of  multimedia  technology  is  not  itself  the  factor  that 
influences  learning,  and  that  “measured  gains  in  studies  of  the  instructional  uses  of 
multiple  media  are  most  likely  due  to  instructional  methods  such  as  interactivity  that  can 
be  used  with  a  variety  of  single  and  multiple  media.”  Alessi  and  Trollip  (1991)  report 
that  “research  aimed  at  proving  that  computer-based  instruction  is  better  than  using  a 
book,  teacher,  or  some  other  traditional  method  has  shown  only  a  small  effect  in  favor  of 
computer-based  instruction.” 

Research  conducted  at  the  Naval  Postgraduate  School  on  a  system  similar  to 
IDCTT  (RPL)  indicated  that  students  were  receptive  to  the  training  and  felt  as  if  they  had 
learned  more.  In  particular,  Johnson  (1994)  stated  that,  as  found  by  a  study  of  the  IDCTT 
(Damage  Control  Assistant)  system,  that  “the  IDCTT  trainer  is  a  highly  effective  training 
aid  in  a  shore  based  environment.”  In  the  same  study,  Johnson  (1994)  reports  that  when 
comparing  actual  performance  scores  obtained  by  students  using  traditional  training 
methods  with  those  obtained  using  IDCTT,  “the  IDCTT  is  clearly  the  more  effective 
training  medium” 

D.  IDCTT  (RPL)  SYSTEM  DESCRIPTION 

The  IDCTT  (RPL)  is  a  U.S.  Navy-developed  software  program  that  provides 
realistic  integrated  shipboard  casualty  control  training  (“IDCTT  Repair  Party  Leader”, 
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N86D  pamphlet).  This  system  utilizes  a  large  database  and  numerous  video  clips  to 
provide  the  trainee  with  a  very  realistic  training  environment.  In  the  system’s  current 
configuration,  the  user  is  presented  with  an  opening  screen  that  allows  him  or  her  to 
select  from  a  few  scenarios,  including  damage  resulting  from  a  collision  while  underway 
and  damage  resulting  from  a  collision  while  at  anchor  in  a  busy  strait.  All  scenarios  are 
designed  around  the  characteristics  and  capabilities  of  the  ARLEIGH  BURKE-class 
guided  missile  destroyer  (DDG-51).  Figure  1  provides  a  sample  view  that  would  be 
presented  to  the  user.  In  this  view,  the  user  (Le.  the  Repair  Party  Leader)  is  able  to 
observe  messages  sent  by  his  investigators,  as  seen  in  the  lower  left  section  of  the 
display.  In  Figure  1,  the  investigator  has  reported  to  Repair  3  that  at  time  1607,  flooding 
water  is  four  feet  deep  in  compartment  4-370-5-A,  Supply  Storeroom  Number  3.  All 
messages  sent  and  received  during  the  IDCTT  scenario  are  generated  in  the  format 
prescribed  by  NWP-3-20.31,  Surface  Ship  Survivability,  ensuring  continuity  of  training. 
Each  time  a  message  is  received  at  the  Repair  Locker,  the  “New  Report”  pushbutton 
above  the  window  flashes  red,  prompting  the  user  to  read  the  new  message.  By  using  the 
arrow  button  on  the  lower  part  of  this  window,  the  user  can  page  through  all  messages, 


Figure  1.  Sample  IDCTT  Display  (from  OPNAV  N86D  pamphlet). 
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reviewing  status  and  progress  of  damage,  as  well  as  subsequent  efforts  to  control  the 
effects  of  the  damage.  The  video  clips  are  presented  in  the  upper  left-hand  portion  of  the 
window.  In  the  case  presented  in  Figure  1,  the  Repair  Party  “phone  talker”  is  (verbally) 
reporting  the  content  of  the  message  presented  in  the  window.  The  ‘'phone  talker”  will 
not  report  the  substance  of  the  message  to  the  RPL  until  the  “New  Report”  pushbutton  is 
depressed,  but  will  inform  the  RPL  that  he  is  in  receipt  of  a  new  report.  The  longer  the 
RPL  waits  to  ask  for  the  report,  the  more  insistent  the  “phone  talker”  will  become.  This 
mimics  the  real  life  flow  of  information  from  the  On-Scene  Leader  (OSL)  to  the  Repair 
Party  Leader,  providing  for  enhanced  realism  and  increased  pressure  for  the  RPL  to  act. 

In  the  right-hand  section  of  the  display,  the  user  is  generating  a  message  to  the 
On-Scene  Leader  in  response  to  the  report  of  flooding  'received  at  1607.  In  this  message, 
the  Repair  Party  Leader  is  directing  the  OSL  to  establish  flooding  boundaries  to  control 
the  effects  of  the  damage  and  prevent  progressive  flooding.  When  the  message  is  ready 
for  transmission  to  the  scene,  the  user  will  have  identified  primary  and  secondary 
flooding  boundaries  both  forward  and  aft  of  the  affected  compartment,  using  the  “pull¬ 
down”  menu  keys  in  the  window.  These  “pull-down”  menus  provide  a  list  of  frame 
numbers  from  which  the  RPL  may  select  to  designate  as  a  boundary.1  Selection  of  an 
incorrect  or  inappropriate  frame  will  result  in  complications  later  in  the  scenario.  When 
the  message  is  complete,  the  user  will  transmit  it  using  the  “Send”  action  button  at  the 
bottom  of  the  window.  Actions  previously  directed  by  the  Repair  Party  Leader  are 
available  for  review  by  selecting  the  “Ordered  Actions”  pushbutton  on  the  message 
dialogue  window. 

Other  features  of  the  display  include  the  stability/firemain  and  information 
sections.  As  can  be  seen  in  Figure  1,  the  stability/firemain  section  provides  visual  output 
of  one  of  the  two  available  inclinometers.  Currently,  the  RPL  has  selected  to  view  the 
heel  inclinometer,  indicating  the  list  of  the  ship  as  a  result  of  the  damage  situation  in 
progress.  He  or  she  could  select  trim  information  by  using  the  mouse  to  click  on  the 

1 A  frame  is  an  athwartships  strength  member  of  a  ship’s  hull,  numbered  from  the  bow  aft.  Frame  numbers 
are  used  as  reference  points  to  locate  fittings  and  compartments  (Noel  and  Beach,  1977).  In  the  design  of  a 
ship,  there  are  special  frames  at  regular  intervals  that  identify  the  location  of  primary  watertight  and 
smoketight  boundaries.  These  particular  frames  are  the  ones  normally  selected  try  the  RPL  at  which  to 
establish  fire  or  flooding  boundaries. 
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Trim  action  pushbutton,  resulting  in  the  display  of  another  inclinometer.2  This 
information  is  important  to  the  RPL  as  it  provides  a  visual  indication  of  the  ship’s 
stability  condition.  The  gauges  on  the  right  side  of  this  window  provide  firemain 
pressure  indications  for  the  port  and  starboard  loops  of  the  DDG-51  firemain  system  In 
the  center  of  the  display,  the  RPL  has  the  option  of  selecting  specific  information 
regarding  one  of  several  topics.  In  this  instance,  the  RPL  is  viewing  the  “Flood/Hull 
Damage”  menu.  From  this  menu,  other  actions  are  available  including  valve  isolation, 
establishment  of  boundaries  and  dewatering  options.  Each  menu  presents  a  list  of  options 
for  the  RPL  based  on  the  general  title  of  the  section.  When  the  user  selects  a  subtopic,  a 
window  "pops  up,”  allowing  the  RPL  the  option  of  directing  the  OSL  to  take  some 
action.  The  flooding  boundary  establishment  window  displayed  in  Figure  1  provides  an 
example  of  what  the  RPL  will  see  when  a  subtopic  is  selected.  If  he  were  interested  in 
undertaking  dewatering  of  the  affected  space,  the  RPL  would  be,  for  example,  offered 
various  pump  options  along  with  several  other  options. 

Also  available  to  the  Repair  Party  Leader  via  the  IDCTT  are  damage  control 
templates,  organization  charts,  communications  information  and  ship’s  characteristics. 
Each  of  these  items  are  required  components  of  the  damage  control  information  library 
maintained  in  the  Repair  Locker.  The  damage  control  templates,  for  example,  provide 
the  RPL  with  the  location  of  structural  fire  and  flooding  boundaries  as  designed  in  the 
ship’s  construction.  This  information  allows  the  RPL  to  make  rapid  and  accurate 
decisions  regarding  the  placement  of  personnel  and  equipment  for  controlling  the  spread 
of  fire  and  flooding.  Additionally,  information  is  readily  available  regarding  the  location 
and  content  of  such  critical  spaces  as  weapons  magazines,  hazardous  materials  storage 
facilities,  and  flammable  liquid  storerooms,  as  well  as  magazine  sprinkler  systems,  Halon 
protected  spaces,  and  other  installed  firefighting  assets.3 


2  List  is  defined  as  the  inclination  of  a  ship  about  a  fore  and  aft  axis,  while  trim  is  the  fore  and  aft 
inclination  of  the  ship  around  an  athwartships  axis  (Noel  and  Beach,  1977). 

3  Halon  is  a  firefighting  agent  that  can  be  employed  from  a  remote  location  This  system  is  generally 
installed  in  engineering  spaces,  flammable  liquid  storerooms,  and  paint  mixing  and  issue  rooms. 
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E.  PRIMARY  GOAL  OF  IDCTT  (RPL) 


The  intent  of  IDCTT  (RPL)  is  to  provide  the  student  with  practice  in  collecting 
and  processing  information,  coordinating  responses,  and  maintaining  the  overall  damage 
control  picture.  These  skills  are  critical  to  the  performance  of  the  Repair  Party  Leader  as 
an  individual,  and  development  of  these  abilities  will  enhance  the  overall  combat 
readiness  of  the  unit.  The  skills  mentioned  above  are  difficult  to  learn  outside  of  actually 
imposing  damage  control  scenarios  and  then  evaluating  the  Repair  Party  Leader’s  actions 
and  responses.  IDCTT  provides  the  RPL  candidate  with  unlimited  opportunities  to  “test 
his  or  her  mettle”  against  the  computer,  practicing  personnel  and  equipment  management, 
prioritization  of  effort  and  event  sequencing.  Feedback  at  the  end  of  each  scenario  allows 
the  RPL  to  determine  his  or  her  progress,  identify  areas  of  weakness,  and  focus  on 
concepts  requiring  remediation.  Additionally,  the  ship’s  Damage  Control  Training  Team 
will  have  access  to  these  training  reports,  enabling  them  to  pinpoint  areas  of  weakness 
that  can  be  corrected  via  tailored  training  sessions. 

F.  FEATURES  AND  BENEFITS  OF  IDCTT  (RPL) 

IDCTT  (RPL)  has  several  embedded  features.  First,  the  system  offers  multiple 
scenarios  including  flooding,  stability,  fire,  structural  damage,  pipe  ruptures  and 
personnel  casualties.  This  software  also  provides  the  support  necessary  to  conduct 
simultaneous  interactive  team  training.  The  ability  to  modify  scenarios  at  the  local 
command  level  enables  the  ship’s  damage  control  training  team  to  tailor  training  to 
specifically  target  areas  of  weakness  identified  during  drills  or  other  training  sessions. 
IDCTT  (RPL)  provides  immediate  student  performance  evaluation  and  feedback, 
reinforcing  the  training  received  by  the  user.  Finally,  cross-platform  connectivity 
between  Windows-,  Unix-  and  OS2-based  operating  systems  allows  for  system 
flexibility.  (Chief  of  Naval  Operations  (OPNAV  N86D)). 

Numerous  potential  benefits  are  available  through  use  of  IDCTT  (RPL). 
Currently,  the  system  supports  training  requirements  including  PQS  and  Fleet  Exercises 
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and  promotes  standardization  of  fleet  training.  IDCTT  (RPL)  improves  decision¬ 
making,  teamwork  and  communications  skills,  reduces  instructor  and  team-training 
workload,  and  supports  flexible  training  schedules.  The  system  will  promote  cross¬ 
training  of  key  watchstanders  by  making  available  an  educational  tool  that  is  fun  and  easy 
to  use.  Since  the  system  allows  for  student  performance  evaluation  and  tracking,  the 
qualification  process  can  be  standardized  throughout  the  fleet,  resulting  in  a  greater 
transfer  of  knowledge  from  platform  to  platform 

One  of  the  great  difficulties  facing  the  leadership  aboard  ship  is  ensuring  the 
adequate  training  and  cross-training  of  key  members  of  the  organic  damage  control 
organization.4  This  newly-devised  system  will  allow  individual  members  of  the  team  to 
pursue,  either  with  or  without  supervision,  knowledge  of  ship’s  systems,  proper  event 
sequencing,  intelligent  application  of  assets  based  on  priority  and  other  important  aspects 
of  damage  control  without  requiring  the  participation  of  every  member  of  ship’s 
company.  This  is  a  definite  improvement  over  the  current  methods  required  for  training, 
and,  in  the  era  of  reduced  shipboard  manning,  will  result  in  enhanced  readiness  aboard 
the  ship. 

G.  IDCTT  (RPL)  TEACHING  POINTS 

The  IDCTT  (RPL)  software  is  designed  to  address  doctrine,  techniques,  and 
procedures  provided  in  various  Navy  damage  control  publications  including  the 
following: 

•  Surface  Ship  Survivability  (NWP  3-20.31) 

•  Practical  Damage  Control  (Naval  Ships  Technical  Manual,  Chapter  079) 

•  Repair  Party  Manual  (COMNAV  SURFLANTIN  ST  3541. 1C  / 
COMNAVSURFPACINST  3541.4B) 


4  Cross  training  refers  to  the  activity  of  preparing  each  member  of  the  Repair  Locker  to  be  able  to  assume 
more  than  one  role  in  the  locker.  This  redundancy  in  training  results  in  the  Repair  Locker  being  able  to 
sustain  casualties,  and  continue  to  execute  it’s  damage  control  function  without  loss  of  continuity. 


11 


•  Firefighting  Ship  (Naval  Ships  Technical  Manual,  Chapter  555,  VI) 

•  Repair  Party  Leader  Personnel  Qualification  Standard  (NAVEDTRA 
43119-G,  Watchstation  318). 

This  coverage  has  been  accomplished  by  establishment  and  incorporation  of 
teaching  points  into  each  scenario  developed  for  IDCTT  (RPL).  A  complete  list  of 
teaching  points  is  provided  as  Appendix  A.  As  an  example,  one  of  the  teaching  points 
addressed  in  a  fire  scenario  is  the  concept  of  establishing  fire  boundaries.  The  applicable 
teaching  point  states  that  “any  physical  barrier  can  be  a  fire  boundary.  Ideally,  fire 
boundaries  are  the  bulkheads,  deck,  and  overhead  surrounding  the  fire.  Secondary 
boundaries  are  generally  set  at  watertight  subdivisions  or  airtight  boundaries.”  The 
reference  for  this  point  is  provided  as  Surface  Ship  Survivability  (NWP  3-20.31).  In  this 
example,  failure  of  the  RPL  to  order  fire  boundaries,  or  order  inappropriate  fire 
boundaries  will  result  in  scenario  events  cascading  to  a  predetermined  “kill  point,”  a  stop 
in  the  software  sequence  from  which  the  RPL  cannot  recover,  and  the  ship  is  lost  due  to 
catastrophic  damage. 

Once  the  scenario  is  completed,  the  student  is  provided  with  a  brief  verbal  and 
visual  critique  of  his  or  her  performance  during  the  drill.  This  evaluation  provides 
information  on  incorrect  procedures,  steps  taken  out  of  sequence,  and  other  areas  of 
difficulty.  The  assessment  also  provides  a  list  of  references  that  the  trainee  may  review 
prior  to  undertaking  a  follow-on  scenario. 
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in.  THE  REPAIR  PARTY  LEADER 


Effective  damage  control  is  clearly  an  important  factor  in  the  survivability  of  a 
ship  at  sea.  This  effort  plays  an  important  role  in  the  ability  of  the  ship  to  sustain 
damage,  reduce  the  impact  of  the  damage  on  vital  combat  systems  and  maintain  the 
ship’s  ability  to  continue  its  primary  mission.  An  important  aspect  of  how  damage 
control  contributes  to  surface  ship  survivability  is  the  requirement  to  “improve  the 
capabilities  and  skills  of  personnel  to  enable  them  to  expeditiously  handle  recovery  from 
combat  scenarios  or  accidents  and  to  promptly  restore  vital  ship’s  systems.”  (NWP  3- 
20.31,  paragraph  1.1) 

The  effective  control  of  damage  in  a  shipboard  environment  requires  the 
participation  of  personnel  from  every  echelon  of  the  ship’s  manning  structure.  A  typical 
shipboard  damage  control  organization  is  as  presented  in  Figure  2.  As  the  senior  person 
in  the  locker,  the  Repair  Party  Leader  is  in  a  position  requiring  expertise  and  knowledge 


Figure  2.  Shipboard  Damage  Control  Organization  (Typical)  (from  NWP  3-20.31,  p.  2-12). 
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in  the  leadership  and  management  of  personnel  assets,  as  well  as  a  thorough  knowledge 
of  damage  control  theory,  systems  and  fundamentals.  Specifically,  as  stated  in  Surface 
Ship  Survivability  (NWP  3-20.31),  the  duties  and  responsibilities  of  the  Repair  Party 
include: 

•  Maintenance  of  stability  and  buoyancy. 

•  Maintenance  of  the  ship’s  structural  integrity  and  maneuverability. 

•  Making  repairs  to  electrical  and  sound-powered  telephone  circuits. 

•  Administering  first  aid  and  transporting  injured  personnel  to  battle 
dressing  stations  without  seriously  reducing  the  DC  capabilities  of  the 
Repair  Party. 

•  Being  familiar  with  the  assigned  area  and  related  DC  systems  and 
equipment. 

There  are  additional  functions  that  lie  within  the  purview  of  specific  repair 
parties,  but  they  will  not  be  addressed  in  this  thesis. 

A.  REPAIR  PARTY  LEADER  TRAINING  PROGRAM 

1.  Current  Practice 

Today,  training  for  candidate  Repair  Party  Leaders  occurs  at  Fleet  Training 
Centers  in  San  Diego,  California  and  Norfolk,  Virginia.  The  course.  Damage  Control 
Repair  Party  Leader  (K-495-0040),  involves  10  days  of  instruction  and  covers  a  broad 
range  of  topics.  The  course  mission  is  to  “train  personnel  in  pay  grades  E-5  and  above  in 
advanced  damage  control  theory  and  techniques  so  that  they  may  fill  Repair  Party  Leader 
billets  in  the  shipboard  damage  control  organization,  and  to  facilitate  the  proper 
management  of  repair  party  personnel  in  casualty  situations  under  all  shipboard  readiness 
conditions.”  (Fleet  Training  Center,  1993) 
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The  Repair  Party  Leader  course  is  divided  into  two  phases,  each  of  which  is  five 
days  in  length.  Both  phases  involve  classroom  lecture,  written  “practicals”  and  hands-on 
training  evolutions.  Phase  II  (second  week)  of  the  course  focuses  on  aspects  of 
Chemical,  Biological  and  Radiological  Defense.  This  thesis  will  study  the  impact  IDCTT 
(RPL)  has  on  student  knowledge  gained  in  Phase  I. 

2.  Repair  Party  Leader  School  Instructors 

Course  instructors  are  highly  qualified  senior  enlisted  personnel  (Chief  Petty 
Officer  or  above)  with  vast  experience  in  shipboard  damage  control.  While  on  sea  duty, 
each  instructor  has  earned  the  Enlisted  Surface  Warfare  Specialist  designation,  indicating 
the  level  of  expertise  that  that  individual  brings  to  the  job.  Prior  to  becoming  an 
instructor,  the  candidate  must  successfully  complete  the  Instructor  Training  course.  This 
course  is  the  Navy’s  primer  in  presenting  educational  techniques  to  the  instructor  prior  to 
them  interfacing  with  die  student.  Additionally,  Fleet  Training  Center  San  Diego  Repair 
Party  Leader  instructors  must  successfully  complete  a  local  certification  program  prior  to 
teaching  a  class. 

3.  Instruction  at  the  Repair  Party  Leader  School 

Instruction  at  the  school  is  presented  primarily  in  the  form  of  lecture  and 
equipment  demonstration.  Class  size  is  limited  to  an  enrollment  of  36  students,  but  the 
average  class  has  approximately  25  students  in  attendance.  During  the  period  of 
instruction,  students  are  exposed  to  information  addressing  Damage  Control 
Organization,  Communications,  Repair  Locker  Coordination,  Biological  and  Chemical 
Warfare  Defense,  Radiological  Defense  and  Radiological  Plotting.  As  part  of  the 
training,  students  are  required  to  complete  several  written  practical  exercises,  as  well  as 
spend  one  day  in  a  damage  control  simulator  (“wet  trainer”)  and  one  period  in  a 
“confidence  chamber”  where  they  are  taught  the  proper  use  and  capabilities  of  the  Navy’s 
gas  mask.  The  first  week  of  training.  Phase  I,  provides  in-depth  exposure  to  various  areas 
of  damage  control,  including  fire  fighting,  flooding  effects  and  stability.  Emphasis  is 
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placed  on  developing  base  skills  in: 


•  repair  locker  administration  and  management 

•  damage  control  communications 

•  plotting  and  symbology 

•  plugging,  patching  and  shoring 

•  other  general  DC  topics 

A  student  meets  graduation  requirements  by  scoring  80%  or  higher  on  two  written 
examinations.  The  first  examination,  given  at  the  end  of  week  one,  tests  the  student’s 
knowledge  of  general  damage  control  concepts  and  procedures,  while  the  second 
examination  covers  aspects  of  Chemical,  Biological  and  Radiological  (CBR)  Defense. 

By  the  limited  nature  of  the  course,  as  well  as  restricted  facilities,  the  student  has 
a  very  limited  opportunity  to  actually  coordinate  the  activities  of  a  Repair  Locker, 
manage  personnel  and  equipment,  or  exercise  decision-making  techniques.  It  is  thought 
that  use  of  IDCTT  (RPL)  as  an  educational  augment  at  the  school  will  enhance  this 
opportunity,  and  result  in  an  improved  graduate,  one  more  ready  to  contribute  to  his  or 
her  ship’s  damage  control  readiness. 

4.  Written  Examinations  Administered  at  Repair  Party  Leader  School 

The  written  examination  is  one  of  the  main  evaluative  mechanisms  used  in  the 
RPL  course  to  determine  a  student’s  exit  level  of  knowledge.  The  test  is  an  exercise  in 
memorization  of  general  damage  control  idiom  and  discrete  activities  (e.g.  pipe  patching 
or  construction  of  shoring),  and  does  not  provide  any  indication  of  the  student’s  level  of 
preparedness  to  actually  lead  a  Repair  Locker  through  an  active  damage  control  scenario. 
This  method  of  assessment  works  well  in  determining  whether  the  student  meets  the 
course  graduation  requirements.  However,  it  is  a  common  misperception  in  the  fleet  that 
RPL  school  graduates  have  spent  a  good  deal  of  time  actually  managing  and  directing  the 
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efforts  of  a  Repair  Locker  while  at  the  school.  This  leads  the  ship’s  senior  damage 
control  leadership  to  overestimate  the  ability  the  graduate  possesses  when  he  or  she 
returns  to  the  ship,  and  creates  frustration  for  both  the  RPL  candidate  and  supervisory 
personnel  while  the  trainee  is  pursuing  final  qualification  as  Repair  Party  Leader. 

B.  PERSONNEL  QUALIFICATION  STANDARDS 

For  many  years,  the  Navy  has  relied  upon  Personnel  Qualification  Standards 
(PQS)  to  provide  a  “map”  for  a  candidate  working  towards  completing  a  qualification  for 
a  specific  watch  station.  These  standards  provide  the  Sailor  with  the  minimum 
requirements  he  or  she  must  meet  in  order  to  become  qualified  in  the  watch  station  in 
question.  The  PQS  will  indicate  prerequisite  qualifications  that  must  be  earned  in  order 
to  begin  the  next  program,  and  decomposes  the  qualification  process  into  Fundamentals, 
Systems  and  Watch  Station  tasks. 

NAVEDTRA  43119-G  (Chief  of  Naval  Education  and  Training)  provides  the 
applicable  standard  to  which  all  Repair  Party  Leaders  must  comply  in  order  to  become 
qualified  in  that  billet.  Prior  to  commencing  the  qualification,  the  Sailor  must  have 
completed  the  prerequisites,  which  include  the  On-Scene  Leader  watchstation  and 
fundamentals  of  Stability.  Of  course,  the  OSL  PQS  also  has  prerequisites,  so  the 
qualifications  build  on  one  another.  From  this  PQS,  it  is  clear  that  the  focus  of  the 
training  is  based  on  coordination  of  Repair  Party  efforts  in  combating  the  various  types  of 
damage  that  can  occur  in  a  ship.  Based  on  a  comparison  of  the  RPL  Course  syllabus  and 
RPL  PQS,  it  would  appear  that  the  school  does  not  provide  the  hands-on  opportunity  that 
the  candidate  requires  in  order  to  complete  the  qualification,  leaving  that  aspect  of  the 
training  to  the  shipboard  damage  control  training  organization.  This  burden  can  be 
greatly  reduced  through  use  of  IDCTT  (RPL)  as  a  vehicle  to  allow  the  RPL  candidate  to 
train  individually  and  complete  the  various  PQS  requirements.  As  stated  earlier,  the 
damage  control  leadership  reviewing  trainee  performance  reports  can  maintain  quality 
control  by  generating  tailored  training  to  address  areas  of  weakness. 
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IV.  METHODOLOGY 


For  this  research,  each  student  involved  was  given  the  option  of  not  participating 
in  the  study.  Of  the  73  students  attending  the  school  during  the  three-month  test  period, 
all  opted  to  participate  in  the  experiment.  Prior  to  taking  the  pre-test  on  the  first  day  of 
the  course,  the  students  received  a  standard  briefing  regarding  the  purpose  and  goal  of  the 
study. 


This  pre-test  is  given  in  support  of  research  that  is  being  done  regarding 
the  possible  incorporation  of  interactive  courseware  in  the  training  process 
at  this  school  Your  participation  in  this  study  is  purely  voluntary,  but  you 
may  have  fun  and  really  learn  something!  This  test  is  given  in  order  to 
determine  the  baseline  level  of  knowledge  each  student  has  upon  enrolling 
in  the  course.  Your  score  on  this  exam  will  have  no  impact  on  the 
remainder  of  the  course.  The  questions  for  this  test  will  be  multiple 
choice,  true- false,  and  fill-in.  Circle  the  correct  answer  or  fill  in  the  blank 
as  required. 


After  taking  the  examination,  students  were  given  a  brief  overview  and 
demonstration  of  the  IDCTT  (RPL)  system  which  was  installed  at  Fleet  Training  Center, 
San  Diego,  for  the  duration  of  the  experiment. 


A.  EXPERIMENTAL  DESIGN 


In  order  to  determine  the  effectiveness  of  the  IDCTT  (RPL),  an  experiment  using 
the  Untreated  Control  Group  Design  with  Pretest  and  Posttest  as  described  by  Cook  and 
Campbell  (1979)  was  used. 

1.  Assignment  of  Students 

a.  March  and  April  Classes 

In  this  experiment,  as  in  any  other,  the  random  assignment  of  students  in 
the  Repair  Party  Leader  classes  to  either  the  control  or  treatment  group  was  an 
important  phase.  Randomness  in  assignment  of  subjects  to  the  two  groups  was  critical  in 
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ensuring  the  equivalence  of  the  groups.  Clearly,  assignment  of  subjects  to  one  or  the 
other  groups  by  means  of  a  truly  random  method  (i.e.  coin  toss)  would  be  optimal. 
However,  in  the  case  of  this  experiment  and  based  on  experience  with  damage  control 
training  in  die  fleet,  it  was  decided  that  several  specific  variables  were  considered 
important  enough  to  include  them  in  the  process  of  assigning  students  to  groups  (Winkler 
and  Polich,  1990).  The  groups  were  designated  as  indicated  below: 

•  Control  Group.  This  group  consisted  of  members  of  the  RPL  School  class 
that  were  not  exposed  to  IDCTT  (RPL) 

•  Treatment  Group.  This  group  was  comprised  of  the  remainder  of  the  RPL 
School  class,  those  who  have  their  training  augmented  by  use  of  the 
IDCTT  (RPL) 

To  assure  the  random  distribution  of  students,  several  factors  were  considered  prior  to 
making  assignment  of  a  particular  student  to  a  group. 

•  Sea  duty  experience 

•  Special/advanced  qualifications  held  (ie.  ESWS) 

•  Gender 

•  Paygrade  (officer  and  enlisted), 

b.  May  Class 

A  different  approach  was  taken  with  regard  to  assigning  students  to  the 
control  group  for  the  May  class.  Volunteers  to  form  the  treatment  group  were  solicited 
from  the  class  in  order  to  test  the  hypothesis  that  willing  subjects  would  perform  better 
than  subjects  selected  at  random.  In  its  shipboard  application,  IDCTT  (RPL)  will  be 
available  to  all  members  of  ship’s  company.  These  sailors  will  be  able  to  use  the  system 
at  their  discretion  in  order  to  enhance  their  knowledge  of  ship’s  systems  and  damage 
control  theory,  as  well  as  improve  individual  decision-making  ability.  For  these 
reasons,  the  aspect  of  vohmteerism  as  a  means  of  capturing  the  essence  of  motivation 
as  a  factor  influencing  learning  was  adopted  for  this  class.  While  certainly  not  random,  it 
was  thought  that  this  approach  would  provide  some  useful  results. 
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2.  Entrance  Data 

Entrance  data  (Appendix  B)  was  obtained  from  each  student.  By  collecting  this 
information,  it  was  posable  to  split  the  class  in  such  a  manner  that  approximately  half  the 
women  assigned  were  placed  in  the  treatment  group,  while  the  other  half  were  designated 
to  the  control  group.  Similar  divisions  were  made  with  respect  to  paygrade,  special 
qualifications  held,  and  sea  duty  experience.  For  obvious  reasons,  this  information  was 
not  considered  in  the  case  of  the  May  class.  Details  regarding  the  composite  population, 
as  well  as  demographics  of  the  control  and  treatment  groups  are  presented  in  section  B.  1 
of  this  chapter  and  Chapter  V.A. 


3.  Implementation  of  the  Experiment 

With  the  exception  of  the  assignment  of  students  to  the  treatment  group 
mentioned  in  section  IV.A.l.b,  the  experiment  was  identically  conducted  for  the  three 
classes.  Both  groups  were  subjected  to  a  written  pre-test  (Appendix  B)  in  order  to 
establish  a  baseline  level  of  knowledge.  This  test  was,  in  feet,  an  alternate  version  of  the 
actual  test  administered  by  the  school  to  determine  whether  the  student  had  met  the 
requirements  for  successful  completion  of  Phase  I  of  the  course.  The  cover  sheet  of  the 
examination  also  included  questions  that  were  used  for  subject  identification  and  data 
gathering  that  would  be  used  in  the  analysis  of  results.  The  control  group  then  proceeded 
through  the  school-prescribed  training  track  (Fleet  Training  Center,  San  Diego,  1993), 
completing  the  course  with  the  normally  given  post-test.  The  treatment  group  also  passed 
through  the  normal  curriculum,  but  was  also  given  standard  scenarios  during  the  course 
requiring  the  use  of  the  IDCTT.  The  normal  post-test  was  then  administered,  completing 
this  group’s  training.  During  week  two  of  the  course,  each  student  sat  before  an  oral 
board  (examination),  similar  to  boards  used  to  qualify  Engineering  Officers  of  the  Watch, 
Officers  of  the  Deck  and  Command  Duty  Officers.  These  oral  boards  were  conducted  by 
two  retired  senior  Naval  officers,  both  with  command  experience.  The  “proctors”  had  no 
knowledge  of  the  subject’s  training  track,  ensuring  an  unbiased  evaluation.  Questions 
asked  at  this  board,  and  provided  as  Appendix  C,  were  designed  to  determine  level  of 
knowledge  based  on  standards  established  by  the  Repair  Party  Leader  Personnel 
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Qualification  Standard  (Chief  of  Naval  Education  and  Training).  This  oral  examination, 
coupled  with  the  end-of-phase  written  examination,  fulfilled  the  requirement  of  the 
experiment’s  post-test. 

B.  STUDY  SAMPLE 

Each  ship  is  required  to  provide  a  mid-grade  to  senior  enlisted  member  or  officer 
to  serve  as  Repair  Party  Leader  in  each  of  the  ship’s  Repair  Lockers.  These  billets  are 
active  only  when  the  ship  is  at  Condition  I,  otherwise  known  as  General  Quarters,  the 
highest  state  of  combat  readiness.  Commonly,  there  are  three  Repair  Lockers  in  a 
cruiser-destroyer  type  vessel,  indicating  the  requirement  to  have  at  least  three  members  of 
ship’s  company  trained  and  qualified  to  fill  the  RPL  billet.  A  requirement  for  serving  as 
an  RPL  is  satisfactory  completion  of  the  formal  Repair  Party  Leader  Course,  as  well  as 
applicable  Personnel  Qualification  Standards.  This  experiment  was  conducted  using 
students  attending  the  Repair  Party  Leader  Course  conducted  at  Fleet  Training  Center, 
San  Diego.  Each  of  these  Sailors  are  prospective  Repair  Party  Leaders  aboard  then- 
assigned  U.S.  Pacific  Fleet-ships,  and  are  representative  of  the  type  individual  who  would 
logically  be  exposed  to  the  training  offered  by  the  interactive  RPL  Trainer.  In  order  to 
attend  this  course,  each  student  must  possess  the  following  basic  prerequisites: 

•  Paygrade  E-5  or  above 

•  Graduate  of  Advanced  Shipboard  Firefighting  School 

•  Completion  of  rudimentary  General  Damage  Control  Personnel 
Qualification  Standards. 

The  Repair  Party  Leader  Course  is  generally  convened  once  per  month,  although  during 
some  months,  no  class  is  held.  Maximum  class  size  is  36  students,  but  on  the  average, 
each  class  has  approximately  25  students  enrolled.  This  experiment  was  conducted  using 
classes  convened  in  March,  April  and  May  1997.  During  this  period,  73  students 
attended  the  class  and  participated  in  the  study. 
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c. 


METRICS 


For  this  research,  several  measures  were  collected.  These  included  the  previously 
mentioned  pre-test  administered  prior  to  any  exposure  to  IDCTT  (RPL)  or  classroom 
instruction,  as  well  as  the  regularly  scheduled  post-course  examination  required  by  the 
school  Finally,  the  oral  examination  was  conducted  for  students  in  both  the  control  and 
treatment  groups.  Data  collected  included  the  percentage  of  correct  answers  (of  30 
questions)  on  the  pre-test,  percentage  of  correct  answers  on  the  post-test  (of  50  questions) 
and  performance  by  the  student  on  the  oral  board.  While  the  first  two  measures  are  clear, 
the  oral  board  requires  some  explanation: 

Oral  board  questions  were  developed  using  information  contained  in  the  Navy’s 
damage  control  publications,  PQS  and  based  on  the  author’s  experience  at  previous 
Repair  Party  Leader  qualification  boards.  The  questions  were  designed  to  determine 
whether  the  students  in  the  treatment  group  were  further  along  than  the  control  group  in 
gaining  the  knowledge  required  to  complete  the  requirements  of  Repair  Party  Leader 
qualification.  Proctors  were  provided  with  specific  instructions  on  how  to  conduct  the 
boards,  as  well  as  guidance  to  be  provided  verbatim  to  the  student  before  the  board  began 
(Appendix  C).  The  student  was  read  the  question  and  then  allotted  one  minute  in  which 
to  provide  an  answer.  Student  responses  that  matched  correct  answers  for  the  questions 
were  marked  on  the  proctor’s  sheet,  and,  when  one  minute  had  passed,  the  next  question 
was  asked  regardless  of  the  status  of  the  previous  question.  Scores  were  calculated  by 
determining  the  percentage  of  correct  responses  provided  out  of  the  49  possible  answers. 
Incorrect  answers  were  not  taken  into  account. 

1.  Variables 

For  this  experiment,  the  dependent  variable  was  the  oral  board  score  recorded  by 
each  participating  Sailor.  Independent  variables  include: 

•  Training  system  (Le.  RPL  School,  or  RPL  School  and  IDCTT)  to  which 
the  Sailor  was  exposed 

•  Sea  duty  experience 
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•  Gender 

•  Level  of  education 

•  Computer  Aptitude  Index  generated  from  portions  of  the  Armed  Services 

Vocational  Aptitude  Battery  (ASVAB)  scores  attained  by  the  Sailor  at  the 
time  of  induction. 

2.  Computer  Aptitude  Index  (CAI) 

In  order  to  capture  some  idea  of  each  participant’s  computer  literacy,  ASVAB 
scores  were  obtained  from  the  Defense  Manpower  Data  Center  in  Monterey,  California. 
As  mentioned  earlier,  data  was  available  for  52  of  the  73  subjects  of  the  research.  In 
order  to  develop  the  CAI,  aspects  of  all  elements  of  die  ASVAB  were  examined  in  order 
to  determine  and  rate  applicability  as  a  measure  of  computer  ability,  problem  solving  and 
decision-making  skills.  An  ASVAB  test  writer  at  DMDC  provided  insight  as  to  the 
intent  of  each  of  these  subtest  areas,  and  helped  in  identifying  the  most  logical  set  that 
would  compose  an  index  of  computer  ability  (R.  Hamilton,  personal  communication, 
June  19,  1997).  Of  these  elements,  three  were  determined  to  be  possible  indicators  of 
these  skills: 

•  Paragraph  Comprehension  (PC) 

•  Arithmetic  Reasoning  (AR) 

•  Coding  Speed  (CS). 

Paragraph  Comprehension  requires  the  individual  to  use  information  to  generate  a 
decision.  This  function  can  be  directly  associated  with  the  idea  that  IDCTT  (RPL) 
provides  the  student  with  reports,  status  and  assets  available,  and  then  requires  the 
student  to  make  decisions  based  on  that  information.  While  the  ASVAB  does  not  address 
damage  control  issues,  the  assessment  of  decision-making  ability  is  generally  similar. 
Arithmetic  Reasoning  evaluates  logical  problem  solving.  A  parallel  between  this  portion 
of  the  ASVAB  and  the  requirements  of  IDCTT  (RPL)  exists  much  in  the  same  way  as 
mentioned  for  the  PC  subtest.  Finally,  the  third  element  of  the  CAI,  Coding  Speed 


24 


measures  the  individuaPs  ability  for  abstract  reasoning.  IDCTT  (RPL)  required  this 
ability  as  the  user  must  be  able  to  postulate  the  downstream  impact  his  or  her  actions  may 
have  on  the  progress  of  the  scenario  in  order  to  successfully  save  his  ship. 

The  DMDC  database  provided  three  versions  of  the  ASVAB  score  for  each  of  the 
52  subjects.  These  scores  include  the  Raw  Score,  Standardized  Subtest  Score  and  the 
Navy  Composite  Score.  The  Standardized  Subtest  Score  was  selected  for  incorporation 
into  the  CAI  as  it  accounts  for  differences  between  the  various  versions  of  the  ASVAB 
administered  at  Military  Entrance  Processing  Stations  (MEPS).  By  design,  the 
Standardized  Subtest  Score  ranges  between  20  and  80,  with  a  mean  of  50  and  standard 
deviation  of  10.  (R.  Hamilton,  personal  communication,  June  19,  1997) 

Based  on  the  information  obtained  from  DMDC,  the  CAI  was  developed  as 
Equation  1.  The  ASVAB  does  not  provide  a  direct  measure  of  an  individuaPs  computer 
prowess.  Therefore,  this  equation  assigns  equal  weight  to  the  three  components  because 

CAI  =  (PC  +  AR+CS)/3  (1) 

there  is  no  way  to  determine  which  of  the  elements  is  the  most  important  in  describing  or 
assessing  an  mdividual  ’•«.  computer  ability. 


V.  DATA  ANALYSIS  AND  RESULTS 


The  nature  of  data  collected  and  the  design  of  the  experiment  suggested  using 
three  different  approaches  for  analysis.  First,  the  complete  set  of  data,  including  all  73 
participants  was  examined.  Second,  data  collected  from  the  May  1997  Repair  Party 
Leader  class  was  analyzed.  As  will  be  shown,  the  difference  between  mean  treatment 
and  control  group  oral  board  scores  for  this  class  was  significantly  higher  than  that  of  the 
other  two  classes,  creating  interest  about  the  data  set.  Finally,  a  censored  data  set,  one 
taking  into  account  information  obtained  from  Defense  Manpower  Data  Center,  was 
studied. 


A.  SAMPLE  DEMOGRAPHICS 

A  wide  variety  of  skill  levels,  ratings  and  experience  was  represented  in  the 
classes  studied.  Additionally,  each  class  was  comprised  of  officers,  enlisted  personnel, 
men,  women,  and  members  of  various  ethnic  groups  and  races.  In  order  to  establish  a 
profile  for  each  participant,  data  regarding  educational  background,  race,  gender  and 
performance  on  standardized  aptitude  examinations  was  collected  from  the  Defense 
Manpower  Data  Center  (DMDC)  master  files.  Unfortunately,  of  the  73  participants, 
DMDC  held  records  on  only  52  persons.  Of  the  21  social  security  numbers  that  produced 
no  matching  record,  12  were  associated  with  the  treatment  group  while  the  remaining 
nine  were  part  of  the  control  group.  Further,  18  of  the  21  records  that  produced  no  match 
in  the  DMDC  database  were  junior  officers.  Lieutenant  or  below.  It  is  possible  that  some 
of  these  officers  never  took  an  aptitude  test,  or  that  they  have  not  been  in  the  Navy  long 
enough  for  the  data  base  to  reflect  any  information.  The  three  remaining  unmatched 
personnel  were  a  Chief  Petty  Officer,  Chief  Warrant  Officer  and  a  Second  Class 
Boatswain’s  Mate.  The  absence  of  information  on  these  Sailors  is  inexplicable.  Level  of 
education  data,  however,  was  obtained  on  all  73  participants.  Table  1  provides 
demographic  information  regarding  the  overall  sample  population. 


27 


Demographics  and  Background 

Total 

Control 

Treatment 

Sample  Size  (N) 

73 

36 

37 

Male  (percent) 

83.56 

86.11 

81.11 

Pay  grade  (rank)  Distribution  (percent) 

E-5 

10.96 

8.33 

13.51 

E-6 

24.66 

27.78 

21.62 

E-7  through  CW04 

24.66 

30.56 

18.92 

0-1  and  0-2 

34.24 

30.56 

37.84 

0-3  and  above 

5.48 

2.77 

8.11 

Educational  and  Aptitude  Characteristics 

Previous  Education  Distribution  (percent) 

Some  college  or  more 

42.46 

38.89 

45.95 

High  School  Diploma 

38.36 

38.89 

37.83 

GED  Certificate 

4.11 

2.78 

5.41 

Less  than  High  School  Diploma 

1.37 

0.00 

2.70 

Unknown 

13.70 

19.44 

8.11 

Career  Characteristics 

Average  Sea  Experience  (months) 

48.49 

Warfare  Qualified  (percent)1 

42.47 

Table  1.  Sample  Demographics. 

Clearly,  the  sample  was  overwhelmingly  male,  with  only  16.5%  of  the 
participants  being  female.  This  is  not  considered  unusual,  as  the  Navy  has  only  recently 
opened  the  majority  of  shipboard  billets  to  females.  Interestingly,  among  the  officers  in 
the  sample,  26%  were  female,  and  most  were  assigned  to  pre-commissioning  units.  Of 
this  group  of  female  officers,  83%  were  assigned  to  the  treatment  group.  Since  the 
Repair  Party  Leader  billets  are  normally  held  by  senior  enlisted  personnel  and  junior 
officers,  it  is  not  surprising  that  almost  59%  of  all  participants  in  the  study  were  between 
the  ranks  of  Chief  Petty  Officer  (E-7)  and  Lieutenant  (junior  grade).  Finally,  the  average 
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sea  experience  of  the  sample  group  shows  the  feet  that  several  of  the  junior  officers  had 
no  previous  shipboard  experience,  while  some  of  the  senior  enlisted  personnel  had 
upwards  of  six  years  at  sea. 

As  mentioned  previously,  three  separate  Repair  Party  Leader  classes  participated 
in  this  study.  Figure  3  shows  the  officer  and  enlisted  composition  of  each  of  the  three 
classes,  as  well  as  the  composite  group  when  considering  all  73  study  participants.  As 
this  graph  indicates,  the  first  two  classes  had  smaller  officer  representation  (30.3%  and 
16.7%  for  the  March  and  April  classes)  than  did  the  May  session  (57.7%).  This 


Figure  3.  Breakdown  by  rank  for  each  RPL  School  class. 

unbalanced  condition  may  influence  results,  and  bears  attention.  Figure  4  provides 
information  regarding  the  distribution  of  officer  and  enlisted  personnel  by  class 
convening  date  as  well  as  group  assignment.  Figure  4  shows  that  class  3,  convened  in 
May,  has  a  much  higher  percentage  of  officer  representation  in  the  treatment  group,  62%, 
as  opposed  to  classes  1  and  2  (50%  and  25%  respectively).  The  effect  of  this  distribution 
will  be  examined  to  determine  impact  on  the  overall  result  of  the  study.  Additionally, 
54%  of  the  May  class  treatment  group  was  composed  of  officers,  as  opposed  to  36%  and 
8%  of  the  March  and  April  sessions.  Considering  these  disparities  may  help  explain 


1  Some  participants  have  earned  qualification  as  a  warfare  specialist  (Surface  Warfare  Officer  or  Enlisted 
Surface  Warfare  Specialist).  Information  regarding  qualification  requirements  for  these  designations  is 
available  in  various  Navy  directives,  including  the  Military  Personnel  Manual  (MILPERSMAN). 
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Class  1  Class  2  Class  3  Total 


Figure  4.  Distribution  of  Officers  and  Enlisted  (by  class  and  group). 

results  obtained. 

In  order  to  utilize  education  level  and  aptitude  examination  information  provided 
by  DMDC,  the  data  set  was  censored  to  include  only  those  personnel  for  whom  this 
information  was  available.  While  this  reduced  the  sample  size  to  52,  it  is  important  to 
determine  if  there  is  any  relationship  between  performance  and  education/aptitude. 
Therefore,  two  presentations  will  be  made;  the  first  neglecting  education  and  aptitude,  but 
considering  the  entire  data  set,  and  a  second  that  considers  these  factors  in  the  analysis. 

Table  2  provides  information  regarding  the  censored  data  set.  As  mentioned 
earlier,  DMDC  was  able  to  provide  information  on  52  of  the  73  study  participants.  The 
information  in  Table  2  is  provided  to  enable  analysis  of  the  impact  the  subject’s  level  of 
education  and  aptitude  as  determined  by  AFQT/ASVAB  tests  administered  upon 
induction  into  the  Navy. 


Demographics  and  Background 

Total 

Control 

Treatment 

Sample  Size  (N) 

52 

27 

25 

Male  (percent) 

84.62 

81.48 

88.00 

Race  Distribution  (percent) 

White 

69.24 

74.07 

64.00 

Black 

15.38 

14.82 

16.00 

Other 

15.38 

11.11 

20.00 

30 


Pay  grade  (rank)  Distribution  (percent) 


E-5 

13.46 

7.41 

20.00 

E-6 

34.62 

37.04 

32.00 

E-7  through  CW04 

30.76 

33.33 

28.00 

0-1  and  0-2 

17.31 

18.52 

16.00 

0-3  and  above 

3.85 

3.70 

4.00 

Educational  and  Aptitude  Characteristics 

Previous  Education  Distribution  (percent)2 

Some  college  or  more 

26.09 

30.43 

21.74 

High  School  Diploma 

54.35 

52.18 

56.53 

GED  Certificate 

4.35 

0.00 

8.69 

Less  than  High  School  Diploma 

2.17 

0.00 

4.35 

Unknown 

13.04 

17.39 

8.69 

AFQT  Score  (mean) 

66.26 

66.74 

65.78 

AFQT  Mental  Category  (percent) 

I  (93-99  percentile) 

15.22 

17.39 

13.04 

II  (65-92  percentile) 

39.13 

43.48 

34.78 

IDA  (50-64  percentile) 

19.56 

8.69 

30.43 

TUB  (31-49  percentile) 

23.91 

26.09 

21.74 

IV  (10-30  percentile) 

2.17 

4.35 

0.00 

V  (1-9  percentile) 

0.00 

0.00 

0.00 

Table  2.  Sample  Population  Demographics  (Censored  Data). 

Table  2  shows  that  the  distribution  between  the  control  and  treatment  groups  is 
fairly  consistent  across  the  spectrum  of  variables.  The  small  percentage  of  subjects  that 
have  earned  a  college  degree  skews  the  reflected  percentages  while  the  raw  numbers  are 
relatively  close.  As  would  be  expected,  the  vast  majority  of  subjects  fell  into  Mental 


2  Data  provided  by  DMDC  indicated  that  of  the  52  subjects,  6  had  unknown  AFQT  scores.  As  such,  these 
personnel  were  excluded  from  the  calculation  of  education  level,  mean  AFQT  and  Mental  Category  figures. 
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Categories  1HA  and  H  The  presence  of  one  Mental  Category  IV  subject  in  the  treatment 
group  is  somewhat  dampened  by  that  group’s  larger  percentage  of  Category  I  personnel. 

Figure  5  provides  a  summary  of  the  censored  data  set.  When  considering  this 
data,  the  officer  population  has  been  reduced  such  that  representation  in  the  March  and 
May  classes  is  fairly  consistent  (35%  and  27%),  while  officers  comprise  only  5%  of  the 


Figure  5.  Officers  and  Enlisted  (by  class)  for  the  Censored  Data  Set. 

April  class.  Figure  6  provides  the  distribution  of  officers  and  enlisted  by  class  and  group 
for  the  censored  data  set.  With  the  officer  population  reduced  from  29  to  11,  one 


Figure  6.  Distribution  of  Officer  and  Enlisted  (by  class  and  group)  for  the 
Censored  Data  Set 


would  expect  there  to  be  an  effect  in  the  distribution  of  officers  throughout  the  three 
classes.  From  this  graph,  it  is  clear  that  the  distribution  of  enlisted  personnel  between  the 
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control  and  treatment  groups  is  consistent  across  all  three  classes.  With  respect  to  the 
officers,  there  are  none  represented  in  the  April  session’s  treatment  group,  and  only  one 
in  the  May  treatment  group.  Over  the  spectrum  of  classes,  however,  the  officers  are 
evenly  divided  between  the  control  and  treatment  groups. 

B.  ANALYSIS  OF  THE  COMPLETE  DATA  SET 

Prior  to  beginning  the  experiment,  it  was  expected  that  the  subject’s  rank  (i.e. 
whether  officer  or  enlisted),  level  of  education,  sea  experience,  advanced  qualifications, 
pre-test  score  and  time  actually  spent  using  the  IDCTT  (RPL)  would  be  important  factors 
in  determining  how  well  the  individual  performed  during  the  post-treatment  oral  board. 
With  this  in  mind,  data  was  collected  for  the  73  participants  to  address  these  variables. 
Appendix  D  provides  the  complete  data  set;  names  and  social  security  numbers  of  the 
participants  are  not  included  because  of  Privacy  Act  restrictions. 

1.  Variable  Selection 

As  an  initial  action,  stepwise  regression  was  employed  in  order  to  assist  in  the 
selection  of  variables.  (Hamilton,  1992)  This  search  strategy  helped  identify  the 
variables  that  had  the  most  significance  in  the  prediction  of  a  student’s  performance 
during  the  Oral  Board.  The  difficulties  Hamilton  (1992)  discusses  regarding  this  method 
were  noted,  but  it  was  decided  that  stepwise  regression  provided  the  simplest  means  of 
determining  the  optimal  variable  combination. 

Appendix  E  provides  the  results  of  stepwise  regression  on  nine  independent 
variables  using  the  “forward  inclusion”  method  provided  by  S-Plus.  In  this  method,  the 
process  starts  with  no  independent  variables  in  the  model,  and  then  first  adds  the  variable 
that  has  the  largest  simple  correlation  with  the  dependent  variable.  At  each  subsequent 
step,  the  independent  variable  that  provides  the  largest  decrease  in  R2  is  added  to  the 
model  (Hamilton,  1992)  As  shown  in  Appendix  E,  the  variables  were  added  in  the 
following  order: 
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1.  Group 

2.  Class 

3.  Pre-Test  Score 

4.  Rank 

5.  ESWS  Qualification 

6.  Education 

7.  IDCTT  (RPL)  Exposure  Time 

8.  Sea  Duty  Experience 

9.  Gender 

Figure  7  provides  a  graphical  representation  of  the  impact  the  addition  of  each  variable 
has  on  the  reduction  in  the  residual  sum  of  squares  (numbers  on  the  x-axis  correspond  to 


size 

Figure  7.  Variable  Selection  via  Stepwise  Regression 


the  independent  variable  listed  above).  As  shown  in  Figure  7,  the  curve  has  a  steep  slope 
until  the  model  contains  five  variables,  and  then  flattens  out,  indicating  that  the  addition 
of  variables  six  through  nine  do  not  provide  a  significant  decrease  in  R2.  This  result 
provides  visual  evidence  that  level  of  education,  time  spent  working  through  scenarios 
using  IDCTT  (exposure  time),  sea  duty  experience  or  gender  did  not  noticeably  influence 
the  subject’s  performance  on  the  Oral  Board. 
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2.  Initial  Data  Analysis 

In  order  to  examine  the  data,  a  tree-based  model  was  developed.  (“S-Phis  Guide 
to  Statistical  and  Mathematical  Analysis”,  1995)  This  model  was  used  to  further  screen 
variables  as  to  their  relative  importance  and  summarizes  the  complex  multivariate  data 
set  generated  by  the  study.  In  this  model,  predicted  values  of  the  independent  variable 
are  made  based  on  various  partitions  of  the  data.  Logical  expressions  are  displayed  at 
each  split,  providing  information  regarding  the  branches  that  follow.  As  is  the 
convention,  the  logical  expression  is  evaluated  as  being  true  for  branches  to  the  left  and 
false  for  right  branches.  Figure  8  provides  the  tree-based  model  generated  by  the 
complete  data  set  using  all  predictor  variables.  This  model  provides  some  interesting 


Exposure  Time  <  6.5  miiutes 


ReTest<  78335  Class  <25 


Baric  <E-7 _  62.59 

fteTest<  66665  Oas<25 

I  I  r  I 

ReTert  <71665 

3436  43.73  | -  |  4402 

Sea  Time  <43  5  mo  6021 

I  I 

49.80  48.16 

Figure  8.  Tree-based  model  (Complete  data  set). 

insight  into  the  data.  Six  members  of  the  treatment  group  spent  less  than  6.5  minutes 
using  the  interactive  system.  Based  on  the  partitioning  algorithm  embedded  in  the  tree- 
based  model  method,  no  significant  differences  exist  between  the  control  and  treatment 
groups  when  exposure  is  less  than  6.5  minutes.  For  all  practical  purposes  then,  the 
branches  on  the  left  side  of  the  root  (IDCTT  exposure  time  less  than  6.5  minutes)  can  be 
considered  representative  of  the  control  group,  while  the  right  side  is  associated  with  the 


Sea  Time  <44 5 mo  Exposure  Time  <17  minutes 


4671  53.97  i  l 

73.81  85.42 
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treatment  group.  One  feature  provided  by  the  tree-based  model  is  a  visual  indication  of 
the  importance  of  each  variable.  As  can  be  seen  in  Figure  8,  the  longest  vertical  branch 
of  the  treatment  group-side  of  the  tree  is  associated  with  the  parent  split  “class.”  The 
longer  the  branch,  the  more  important  the  model  considers  the  variable.  As  for  the 
control  group,  the  most  important  variables  are  pre-test  score,  followed  by  rank. 

a.  Control  Group  Analysis 

With  respect  to  the  control  group,  the  first  partition  in  the  tree  is  based  on 
the  pre-test  score  recorded  by  the  student  (Figure  8).  Students  that  earned  a  score  in 
excess  of  78.335  could  be  expected  to  answer  62.59%  of  the  oral  board  questions 
correctly,  regardless  of  rank.  This  figure  is  higher  (in  most  cases,  substantially  so)  than 
oral  board  scores  earned  by  students  not  doing  as  well  on  the  pre-test.  Mid-grade  enlisted 
students  (paygrade  E-5  and  E-6,  with  respective  rank  indicators  of  1  and  2 )3  scoring 
between  66.665  and  78.335  could  be  expected  to  correctly  answer  43.73%  of  questions 
asked  at  the  oral  board,  while  those  posting  pre-test  scores  less  than  66.665  would 
correctly  answer  only  34.36%.  Within  the  control  group,  the  class  in  which  the  student 
was  enrolled  was  a  factor  for  students  in  paygrades  E-7  through  0-4  (rank  indicators  of  3 
through  5).  For  these  students,  those  in  the  May  RPL  class  (class  3)  scored  44.02%  on 
the  oral  board,  while  those  in  classes  convened  in  March  and  April  (classes  1  and  2, 
respectively)  earned  slightly  higher  scores.  As  shown  in  the  fourth  echelon,  students  in 
the  first  two  classes  scoring  higher  than  71.665  on  the  pre-test  did  significantly  better  on 
the  oral  board.  Even  though  the  algorithm  continues  to  split  the  data,  it  is  evident  that  sea 
time  was  not  a  factor  producing  great  differences  in  expected  oral  board  scores  within  the 
control  group. 


b.  Treatment  Group  Analysis 

As  mentioned  earlier,  the  right  side  of  the  tree  in  Figure  8  represents  the 
model  associated  with  the  treatment  group.  While  much  simpler,  this  portion  of  the  tree 


3  Subjects  in  this  study  consisted  of  personnel  of  paygrade  E-5  (mid-grade  enlisted)  through  0-4  (midgrade 
officer).  In  order  to  simplify  analysis,  integers  representing  the  paygrades  were  assigned  as  follows:  1  - 
E-5,  2  -  E-6,  3  -  E-7  through  W-4,  4  -  0-1  and  0-2,  5  -  0-3  and  above.  An  indicator  of  2.5,  therefore, 
delineates  the  boundary  between  E-6  and  E-7. 
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provides  some  interesting  information  regarding  the  differences  in  the  three  classes.  In 
this  case,  the  algorithm  partitioned  the  data  based  initially  on  the  classes,  dividing  the  set 
into  March  and  April  classes,  and  the  May  class.  Within  the  March  and  April  classes,  sea 
time  was  an  important  factor  in  predicting  performance  on  the  end-of-course  oral  board. 
Those  personnel  with  less  than  44.5  months  of  sea  duty  could  be  expected  to  correctly 
answer  46.71%  of  the  oral  board  questions,  while  those  with  more  time  at  sea  would  be 
expected  to  score  7.26%  higher.  It  is  interesting  to  note  that  sea  time  was  considered  to 
be  the  eighth  most  important  variable  when  stepwise  regression  was  employed.  For 
students  in  the  May  class,  IDCTT  (RPL)  exposure  time  was  the  partitioning  factor.  In 
this  case,  those  students  with  greater  than  17.5  minutes  of  system  contact  time  scored 
1 1.61%  higher  than  did  those  with  less  time  running  damage  control  scenarios.  The  most 
interesting  aspect  of  the  treatment  group- side  of  the  tree  is  the  range  of  expected  oral 
board  scores.  These  scores  range  from  46.71  to  85.42,  a  very  wide  span.  This  variation 
raises  questions  regarding  data  collected  from  the  May  RPL  class,  and  provides  support 
for  the  three-phase  analysis. 

c.  Combined  Analysis 

As  stated  earlier  in  this  chapter,  the  inconsistencies  in  oral  board  performance 
across  the  three  classes  is  of  interest.  Since  the  tree  presented  in  Figure  8  indicates  a 
large  difference  in  performance  between  the  March  and  April  classes  and  that  of  the  May 
class,  a  second  tree-based  model  was  constructed.  In  this  model  (Figure  9),  the  left 
branches  of  the  tree  correspond  to  the  control  group,  while  the  right  side  represents  the 
treatment  group.  In  this  model,  it  is  clear  that  the  May  class  treatment  group  could  be 
expected  to  earn  substantially  higher  scores  on  the  oral  board  than  any  other  group,  while 
control  group  oral  board  scores  are  roughly  equivalent  across  the  three  classes.  As  the 
May  class  treatment  group  was  composed  of  volunteers,  this  striking  result  may  provide 
evidence  supporting  the  belief  held  by  many  educators  that  .motivation  is  the  ultimate 
key  to  success  in  the  classroom.”  (Poirot,  1992) 
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3.  Analysis  of  Variance 

As  presented  earlier,  stepwise  regression  identified  five  significant  independent 
variables.  Based  on  that  regression,  ANOVA  was  conducted  on  a  model  constructed 
using  these  variables,  with  the  oral  board  score  as  the  response  variable.  Appendix  F 
provides  the  results  of  this  analysis,  including  interaction.  A  simple  additive  linear  model 
was  also  constructed  using  the  variables  identified  in  the  stepwise  regression.  As 
discussed  by  Hamilton  (1992),  the  selection  of  variables  using  a  search  strategy  such  as 


G  r(  up  <  0.5 


Figure  9.  Tree-based  model  of  Complete  Data  Set  (Group  only). 

as  stepwise  regression  is  not  without  pitfalls.  Hamilton  states  that  p-values  generated 
using  these  variables  are  “unrealistically  low,”  casting  doubt  as  to  the  meaning  of  the 
significance  of  each  variable.  With  this  in  mind,  Hamilton  recommends  using  p-values 
for  descriptive  value  only.  Therefore,  the  following  analysis  must  be  accepted  with 
caution.  Table  3  shows  that  of  the  five  variables,  group  and  class  are  highly  significant, 
rank  and  ESWS  qualification  are  moderately  significant,  and  pre-test  score  is 
insignificant  This  simple  model  produced  an  R2  value  of  .3309,  indicating  that  the 
model  variables  account  for  only  33%  of  the  variance.  As  shown  in  the  results  of 
ANOVA  (Appendix  F),  of  the  five  variables  in  the  model,  group,  class  and  ESWS 
qualification  are  significant  predictors  of  a  student’s  performance  on  the  oral  board, 


while  pre-test  and  rank  are  insignificant  at  a=0.05.  The  complex  and  uninterpretable 
interactions  revealed  by  ANOVA  suggested  the  adoption  of  a  simple  additive  model 
using  these  three  independent  variables. 


Value 

Std.  Error 

T  value 

Intercept 

12.8303 

11.4069 

1.1248 

0.2647 

Group 

11.3005 

3.2835 

3.4416 

0.0010 

Class 

6.8078 

1.9894 

3.4221 

0.0011 

PreTest 

0.1665 

0.1388 

1.2000 

0.2344 

Rank 

2.9968 

1.6182 

1.8520 

0.0684 

ESWS 

7.2689 

3.9908 

1.8214 

0.0730 

Table  3.  Linear  Model  with  Five  Variables  Model. 

Figure  10  is  a  boxplot  comparison  of  control  and  treatment  group  oral  board 
scores.  From  this  plot,  it  would  appear  that  the  treatment  group  recorded  significantly 


Figure  10.  Boxplot  comparison  of  Oral  Board  Scores  (Complete  Data  Set). 

higher  scores  on  the  post-course  oral  board.  However,  variability  between  the  two 
groups  is  inconsistent.  The  median  score  for  the  treatment  group,  represented  by  the 
white  horizontal  line  across  the  box,  was  57.14,  while  the  control  group  median  score 
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was  48.98.  While  this  is  a  large  difference,  it  is  unclear  as  to  whether  the  difference  was 
the  result  of  the  treatment,  a  combination  of  the  interaction  of  the  independent  variables, 
or  chance.  In  order  to  determine  the  source  of  the  difference,  boxplots  were  constructed 
comparing  the  control  and  treatment  group  oral  board  scores  as  influenced  by  the 
important  independent  variables  identified  by  ANOVA  (class  and  ESWS  qualification). 
As  Figure  1 1  shows,  there  was  greater  variability  for  those  students  previously  ESWS 
qualified,  and  those  students  enrolled  in  the  May  session  (class  3).  The  overlap  of  the 
quartile  boxes  would  indicate  that  there  is  no  difference  between  oral  board  scores  when 
these  variables  are  considered. 


Figure  11.  Boxplots  of  Complete  Data  Set  variability. 

4.  Hypothesis  Testing 

In  order  to  determine  the  effectiveness  of  IDCTT  (RPL)  over  conventional 
schoolhouse  methods  in  the  case  of  the  complete  data  set,  the  following  hypothesis  was 
tested: 


Hc>:  The  mean  oral  board  score  recorded  by  the  control  and  treatment  group  were 
equal. 

versus 

Hi:  The  treatment  group  recorded  a  higher  mean  oral  board  score. 
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As  a  prelude  to  actually  testing  the  null  against  the  alternative  hypothesis,  the 
power  of  the  statistical  test  was  calculated  (Appendix  G).  Power  is  defined  as  the 
probability  of  finding  the  null  hypothesis  to  be  wrong,  given  the  decision  rule  and  the  true 
parameter  value  under  the  alternative  hypothesis  (Larsen  and  Marx,  1986).  In  other 
words,  power  is  a  measure  of  the  sensitivity  of  the  test  to  detect  differences  in  the 
parameters  being  compared.  In  the  case  of  this  analysis,  the  parameter  of  interest  is  the 
mean  of  control  and  treatment  group  oral  board  scores.  Higher  power  over  a  broad  range 
of  hypothesized  differences  between  the  means  would  be  ideal;  of  course,  common  to  all 
power  functions  is  a  reduced  capability  to  detect  differences  as  they  approach  zero. 

For  this  analysis,  it  had  to  be  shown  that  the  variance  of  control  and  treatment 
group  oral  board  scores  was  equal,  hi  order  to  do  this,  a  two-sided  F  test  was  conducted. 
The  following  hypothesis  was  checked  using  this  test: 

Ho:  The  variances  of  the  mean  oral  board  scores  recorded  by  the  control  and 

treatment  group  were  equal. 

versus 

Hi :  The  variances  were  not  equal. 

With  an  F-statistic  equal  to  1.9855,  a  p-value  of  0.045  was  produced.  Based  on  this 
result,  and  at  a  level  of  significance  of  0.05,  the  null  hypothesis  would  be  rejected.  This 
small  margin  of  rejection,  however,  allows  for  the  statement  that  the  variances  were  not 
significantly  different,  and  testing  could  continue. 

With  equal  variance  and  normality  (essentially)  shown,  the  test  statistic  generated 
(Equation  2)  is  distributed  as  a  t  random  variable  with  n  +  m  -  2  degrees  of  freedom 

fsT 

V  n  +  m 
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(where  n  and  m  are  the  sizes  of  the  control  and  treatment  group  samples).  When  sample 
size  is  large,  the  t  distribution  looks  Normal.  As  such,  the  power  functions  developed  are 
Normal  approximations. 

Figure  12  is  the  power  function  developed  using  the  complete  data  set.  As  would 
be  expected,  power  is  lowest  when  the  control  and  treatment  groups  are  hypothesized  to 
be  equal  In  fact,  the  power  at  this  point  is  equal  to  the  level  of  significance  of  the  test,  in 
this  case  a  =  0.05.  What  Figure  12  provides  is  an  indication  of  the  test’s  ability  to  detect 
differences  between  the  means  over  the  range  of  -30  (i.e.  the  control  group  mean  is 
larger)  to  30  (Le.  treatment  group  mean  is  larger).  As  shown,  this  test  has  relatively  high 
power  and  can  be  expected  to  detect  a  ten  percent  difference  between  the  means  with 
probability  approximately  0.40.  As  the  hypothesized  difference  becomes  larger,  the 


Figure  12.  Power  Function  for  Complete  Data  Set. 

power  of  the  test  increases  until  it  approaches  one.  So,  for  a  large  difference  between  the 
control  and  treatment  group  means,  this  test  is  likely  to  detect  a  difference.  Conversely, 
for  small  differences  (less  than  ten  percent),  the  power  of  the  test  is  limited. 

Prior  to  testing  this  hypothesis,  however,  the  data  was  explored  to  determine  the 
nature  of  the  underlying  distribution.  Figure  13  provides  four  plots  used  as  diagnostic 
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tools  with  which  to  begin  the  analysis  of  the  data.  The  top  left  plot  is  a  histogram  of  the 
data.  The  lower  left  plot  is  known  as  a  density  plot.  This  plot  is  essentially  a  smooth 
version  of  the  histogram  and  provides  smooth  estimates  of  population  frequency  or 
probability  density  curves.  As  they  are  similar,  the  density  curve  and  histogram  should 
have  basically  the  same  shape.  However,  differences  in  how  the  software  package 
generates  bin  sizes  can  result  in  differently  shaped  plots.  Based  on  the  histogram  in 
Figure  13,  the  data  appeared  to  be  positively  skewed.  By  applying  a  power 
transformation  (raised  to  the  0.8  power),  the  distribution  of  the  data  was  made  more 
symmetrical  The  histogram  and  boxplot  (Figure  14)  indicate  the  effect  of  the 
transformation.  Based  on  the  visual  evidence  provided  in  Figure  14,  the  transformed 
data  was  assumed  to  be  normally  distributed,  permitting  the  application  of  standard 
statistical  tests  of  the  previously  mentioned  hypothesis,  using  the  transformed  data. 
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Figure  13.  Diagnostic  Plots  for  Complete  Data  Set. 
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To  test  the  hypothesis  that  the  mean  oral  board  scores  recorded  by  the  control  and 
treatment  group  were  equal,  a  two-sample  t  test  was  conducted  on  the  transformed 


Transformed  Oral  Board  Score 
(Complete  Data) 


Transformed  Oral  Board  Score 
(Complete  Data) 


Figure  14.  Diagnostic  Plots  for  Transformed  Data  (Complete  Data  Set). 

data.  This  test  indicated  that  at  a  level  of  significance  of  0.05  and  a  (-statistic  of  3.4026, 
there  was  a  difference  between  the  means  (p-value  =  0.0006),  and  the  null  hypothesis  was 
rejected.  The  treatment  group,  in  feet,  attained  a  mean  oral  board  score  20.26%  higher 
than  that  of  the  control  group  (60.78  to  48.47).  It  could  not  be  measured,  however,  the 
exact  contribution  use  of  IDCTT  (RPL)  had  in  this  learning  benefit. 

5.  Analysis  of  Results 

Since  this  experiment  was  based  on  the  assertion  that  use  of  interactive 
courseware  as  an  instructional  aid  benefits  learning,  the  discovery  that  the  group  the 
student  was  part  of  (i.e.  control  or  treatment)  was  significant  is  not  surprising.  As 
previously  mentioned,  the  treatment  group  used  volunteers  from  the  May  class,  so  the 
oral  board  performance  differences  between  the  classes  could  be  expected.  Table  4 
provides  a  principal  component  analysis  of  the  independent  variables  in  order  to  further 
explain  the  factors  contributing  to  differences  in  oral  board  scores.  From  this  table,  the 
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class  the  student  was  in  accounts  for  64%  of  the  variance  in  the  model.  However,  31% 
of  the  variance  is  explained  by  group  affiliation.  This  information  provides  further 


Standard  Deviation 

Proportion  of  Variance 

Cumulative  Proportion 

Class 

40.25 

0.6375803 

0.6375803 

Group 

27.88 

0.3060058 

0.9435861 

Rank 

11.72 

0.05407455 

0.99766068 

Education 

2.03 

0.001618563 

0.999279238 

ESWS 

0.90 

0.0003188014 

0.9995980396 

Gender 

0.79 

0.0002443316 

0.9998423712 

Sea  Time 

0.44 

0.00007777285 

0.99992014409 

Pre  Test 

0.33 

0.00004408595 

0.99996423004 

Exposure  Time 

(<  6.5  minutes) 

0.30 

0.00003576996 

1.00000000000 

Table  4.  Variance  as  explained  by  Principal  Components. 


evidence  that  use  of  IDCTT  (RPL)  provides  some  educational  benefit  to  the  students  in 
the  class.  However,  the  fact  that  IDCTT  (RPL)  exposure  time  was  inconsequential 
creates  concern  as  to  the  quantitative  advantage  gained  by  students  in  the  treatment 
group.  The  importance  of  the  class  the  student  was  enrolled  in  has  to  be  considered  as  a 
confounding  variable.  Since  this  component  accounted  for  a  majority  of  the  variance  in 
the  model,  it  must  be  addressed.  Analysis  of  data  collected  from  the  May  class  will 
hopefully  clarify  this  issue. 


C.  ANALYSIS  OF  DATA  COLLECTED  DURING  MAY  1997  RPL  SCHOOL 
CLASS 

Techniques  employed  in  examining  the  complete  data  set  were  used  in  exploring 
data  collected  from  the  May  RPL  School  class  (Appendix  D).  As  mentioned  earlier  in 
this  chapter,  obvious  differences  in  oral  board  performance  among  members  of  the 
treatment  groups  necessitated  this  separate  analysis. 
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1.  Variable  Selection 

Stepwise  regression  was  employed  using  the  eight  available  variables.  As 
illustrated  in  Figure  15,  this  regression  identified  six  variables  that  made  important 


size 


Figure  15.  Variable  Selection  via  Stepwise  Regression. 


contributions  to  the  reduction  in  the  residual  sum  of  squares.  As  such,  the  following 
variables  were  used  in  the  construction  of  a  simple  additive  linear  model  and  conduct  an 
ANOVA: 

1.  Group 

2.  Pretest  Score 

3.  Exposure  Time 

4.  Sea  time 

5.  ESWS  Qualification 

6.  Education 

2.  Distribution  of  the  Data 

Figure  16  provides  a  series  of  diagnostic  plots  used  to  determine  the 
characteristics  of  the  underlying  distribution  of  the  data.  The  QQ-plot  and  boxplot 
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provide  an  indication  that  the  data  set  has  light  tails  and  very  few  outlying  points.  The 
information  provided  in  Figure  16  does  not,  however,  provide  much  clarification  as  to  the 
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Figure  16.  Diagnostic  plots  for  the  May  97  Class. 

distribution  of  the  data.  To  further  examine  the  data,  composite  goodness-of-fit  tests 
were  applied.  In  these  tests,  the  parameters  required  (mean  and  standard  deviation)  were 
estimated  from  the  data.  As  a  result  of  parameter  estimation,  the  goodness-of-fit  tests 
were  limited  in  their  ability  to  determine  the  distribution  and  specific  parameters 
underlying  the  data.  In  fret,  composite  goodness-of-fit  tests  can  only  determine  the 
family  of  distributions  to  which  the  data  belongs  (“S-Phis,  Guide  to  Statistical  & 
Mathematical  Analysis”,  1995). 

The  null  hypothesis  for  these  tests  was  that  the  cdf  of  the  data  equaled  the  Normal 
distribution,  with  the  alternative  being  that  they  were  not  equal.  Figure  17  is  a  plot  of  the 
empirical  cdf  and  hypothesized  Normal  cdf.  This  plot  provides  a  foundation  that  the  data 
may  be  assumed  to  belong  to  the  Normal  family  of  distributions,  a  conjecture  that  was 
further  examined  by  goodness-of-fit  tests.  The  first  goodness-of-fit  test  applied  was  the 
one-sample  Kolmogorov- Smirnov  test.  Using  the  estimated  parameters,  the  null 
hypothesis  could  not  be  rejected  (p-value  =  0.5)  at  a  5%  level  of  significance.  In  order  to 
confirm  this  result,  the  chi-square  goodness-of-fit  test  was  applied  to  the  same 
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hypothesis.  Again,  the  results  of  this  test  do  not  permit  the  rejection  of  the  null 
hypothesis  that  the  data  is  from  a  non-specific  Normal  distribution  (p-value  =  0. 1649).  In 
both  cases,  the  p-values  returned  caused  suspicion  about  the  null  hypothesis,  hut  did  not 


Figure  17.  Empirical  and  Hypothesized  Normal  CDF’s. 

allow  for  definite  rejection  at  the  level  of  significance  specified.  In  particular,  the  chi- 
square  test  p-value  was  suspect  as  the  test  was  applied  to  a  small  sample,  when  it  is  most 
effective  for  large  data  sets  (“S-Phis,  Guide  to  Statistical  &  Mathematical  Analysis”, 
1995). 

Based  on  the  visual  evidence  presented  in  Figures  16  and  17,  as  well  as  the  results 
provided  by  the  goodness-of-fit  tests,  the  data  were  assumed  to  be  Normally  distributed 
and  were  analyzed  as  such. 

3.  Initial  Data  Analysis 

As  mentioned  earlier  in  this  section,  group,  pre-test  score,  exposure  time,  sea 
time,  ESWS  qualification  and  education  were  found  by  stepwise  regression  to  be  the 
most  important  factors  with  regard  to  the  May  97  Repair  Party  Leader  School  class 
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(Appendix  E).  A  tree-based  model,  as  discussed  in  Chapter  IV. A.  2,  was  constructed, 
using  all  six  variables.  As  shown  in  Figure  18,  the  partitioning  algorithm  identified  only 


Exposure  Time  <  5  minutes 


Sea  Time  <25.5  mo 


Exposure  Time  <  17  minutes 


52.04 


39.07 


73.81 


85.42 


Figure  18.  Tree-based  model  (May  97  class  data  set). 


two  of  the  variables  as  being  significant  for  analytical  purposes.  The  first  division  was 
made  based  on  IDCTT  exposure  time.  In  Figure  8  the  partitioning  algorithm  split  the 
data  based  on  6.5  minutes  of  exposure  time.  Using  the  May  97  class  data,  this  same 
algorithm  divided  the  data  at  five  minutes  of  exposure.  All  13  members  of  the  treatment 
group  in  die  May  class  had  greater  than  five  minutes  of  use  of  IDCTT  (RPL).  Thus,  as  in 
the  previous  tree  (Figure  8),  the  left  branch  of  Figure  18  represents  the  control  group 
(exposure  time  <  5  minutes),  while  the  right  side  corresponds  to  the  treatment  group.  It  is 
clear  from  Figure  18  that  the  May  class  treatment  group  performed  at  a  much  higher  level 
on  the  oral  board  than  did  the  control  group.  With  regard  to  the  control  group,  sea  duty 
experience  was  a  major  factor  in  influencing  oral  board  performance,  with  the  division 
made  at  just  over  two  years.  In  this  particular  class,  the  treatment  group  scored  an 
average  of  78%  higher  on  the  oral  board  than  did  the  control  group. 
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4.  Analysis  of  Variance 

In  order  to  determine  the  importance  of  the  independent  variables  identified  in  the 
stepwise  regression,  ANOVA  was  conducted.  Appendix  J  provides  the  complete  result 
of  this  procedure,  and  includes  all  interaction  effects.  From  this  ANOVA,  only  group  is 
significant  at  the  0.05  level,  while  pre-test  score  and  ESWS  qualification  would  be 
significant  at  a  10%  level  of  significance.  This  result  was  expected,  as  the  difference  in 
oral  board  scores  between  the  control  and  treatment  groups  was  substantial,  hi  order  to 
provide  for  comparison  with  the  results  attained  by  ANOVA,  a  simple  additive  linear 
model  was  developed  using  the  same  independent  variables.  Table  5  shows  that  at  a  = 
0.05,  group  and  sea  duty  experience  were  significant.  At  a  10%  level  of  significance, 


Value 

Std.  Error 

t  value 

Pr(>|t|) 

Intercept 

35.9033 

13.4969 

2.6601 

0.0155 

Group 

21.6808 

6.0395 

3.5898 

0.0020 

PreTest 

0.1445 

0.1732 

0.8340 

0.4146 

Exp  Time 

0.4137 

0.2088 

1.9809 

0.0623 

SeaTime 

-0.2300 

0.1006 

-2.2857 

0.0339 

ESWS 

13.0991 

6.6580 

1.9674 

0.0639 

Education 

0.6712 

0.8156 

0.8229 

0.4208 

Table  5.  Linear  Model  for  Six  Variables  (May  Data  Set). 


IDCTT  (RPL)  exposure  time  and  ESWS  qualification  were  shown  to  be  important  factors 
in  predicting  oral  board  performance.  The  linear  model  generated  in  the  case  of  the  May 
class  data  provided  a  very  good  fit  and  explained  almost  89%  of  the  variance  in  the  data 
(R2  =  0.8865). 

Principal  component  analysis  was  conducted  on  this  data  set  in  order  to  identify 
the  variables  that  explained  the  majority  of  the  variance.  As  shown  in  Table  6,  the  group 
the  student  was  in  accounts  for  78%  of  the  variance  in  the  model,  while  rank  and 
education  explain  15%  and  7%  respectively.  Combined,  these  three  variables  explain 
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Standard  deviation 

Proportion  of  Variance 

Cumulative  Proportion 

Group 

32.1587715 

0.7773859 

0.7773859 

Rank 

13.9610838 

0.1465130 

0.9238988 

Education 

9.82459602 

0.07255497 

0.99645382 

ESWS 

1.969600738 

0.002916045 

0.999369860 

Gender 

0.7989040141 

0.0004797632 

0.9998496232 

SeaTime 

0.32548378437 

0.00007963362 

0.99992925684 

PreTest 

0.25071729527 

0.00004725049 

0.99997650733 

Exposure  Time 

0.17678575526 

0.00002349267 

1.00000000000 

Table  6.  Variance  as  explained  by  Principal  Components  (May  Class  Data  Set). 


99.6%  of  the  variance  in  the  data.  Of  interest,  this  was  the  only  instance  where 
education  was  a  significant  factor  in  the  linear  model.  Eight  of  13  students  (61.5%)  in  the 
May  RPL  class  treatment  group  had  earned  college  degrees.  This,  coupled  with  the  fact 
that  they  volunteered  to  be  in  the  treatment  group  most  likely  influenced  the  oral  board 
scores,  and  would  explain  the  appearance  of  education  as  a  significant  variable. 


5.  Hypothesis  Testing 

In  order  to  determine  or  quantify  the  impact  on  preparing  a  candidate  Repair  Party 
Leader  gained  by  using  IDCTT  (RPL),  the  hypotheses  generated  in  Chapter  IV.  A.  4  were 
tested.  As  stated  in  Chapter  IV.B.2,  the  dependent  variable  was  assumed  to  be  Normally 
distributed,  enabling  the  use  of  parametric  methods  for  hypothesis  testing.  In  order  to 
satisfy  the  assumption  of  equal  variance  between  control  and  treatment  group  oral  board 
scores,  the  following  hypothesis  was  evaluated,  using  a  two-sided  F-test: 

Hoi  The  variances  of  the  May  class  oral  board  scores  recorded  by  the  control 
and  treatment  group  were  equal. 

versus 


Hi :  The  variances  were  not  equal. 


The  F-test  yielded  a  test  statistic  of 0.853 1  and  a  p-value  of 0.7876.  Based  on  a  level  of 
significance  of  0.05,  the  null  hypothesis  could  not  be  rejected.  Therefore,  for  the 
purposes  of  this  analysis,  the  variances  of  the  control  and  treatment  group  oral  board 
scores  were  assumed  to  be  equal. 

As  discussed  in  Chapter  V.A.4,  a  power  function  was  developed  for  the  May  97 
data  set.  Figure  19  indicates  that,  as  was  the  case  with  the  complete  data  set,  the  test  of 
hypothesis  has  greater  ability  to  detect  larger  differences  between  mean  control  and 


Difference  between  Data  Means 


Figure  1 9.  Power  Function  for  May  97  Data  Set. 

treatment  group  oral  board  scores.  This  curve  was  developed  using  a  range  of 
hypothesized  differences  in  control  and  treatment  group  means  over  the  interval  (-40,  40) 
due  to  the  feet  that  the  actual  difference  is  35.008.  As  Figure  19  shows,  the  power  of  the 
test  to  detect  a  difference  between  the  means  equal  to  20  percent  is  approximately  0.80, 
while  if  the  mean  difference  were  10  percent,  the  power  of  the  test  is  reduced  to 
approximately  25%. 

To  test  the  hypothesis  that  the  control  and  treatment  group  oral  board  mean  scores 
were  equivalent,  a  two- sample  t  test  was  conducted.  This  test  revealed  that,  at  a  =  0.05, 
the  null  hypothesis  was  rejected  (p-value  =  0).  In  order  to  determine  the  true  magnitude 
of  the  difference  between  control  and  treatment  group  mean  oral  board  scores,  the 
hypothesis  was  tested  using  an  arbitrary  value  for  the  difference  in  the  means.  Since  the 
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calculated  difference  is  approximately  35,  the  value  for  the  test  was  established  arbitrarily 
at  42.5.  When  the  null  hypothesis  is  changed  to  test  whether  the  difference  between  the 
treatment  and  control  mean  oral  board  scores  is  greater  than  42.5,  the  null  hypothesis  is 
rejected,  but  with  a  much  higher  p-vahie  (0.0487).  The  true  treatment  group  mean 
(80.06)  is  significantly  larger  than  the  true  control  group  mean  (45.05538),  indicating  that 
for  file  May  class,  the  use  of  IDCTT  (RPL)  may  have  had  a  significant  impact  on  oral 
board  performance. 

6.  Analysis  of  Results 

Clearly,  the  May  class  treatment  group  performed  better  on  the  oral  hoards  when 
compared  with  any  other  group  in  the  study.  The  feet  that  IDCTT  (RPL)  exposure  time 
was  significant  at  the  10%  level  (p-value  =  0.0623)  is  interesting  in  that  this  class  was  the 
only  one  where  use  of  the  system  reflected  any  significant  benefit  to  the  students  in  the 
treatment  group.  Within  the  treatment  group,  it  would  appear  that  exposure  to  more  than 
three  IDCTT  (RPL)  sessions  was  important  in  the  student’s  performance  on  the  oral 
board.  As  shown  in  Figure  18,  a  student  with  more  than  17  minutes  of  time  using  IDCTT 
would  could  be  expected  to  score  nearly  12  points  higher  on  the  oral  board  than  one  with 
less  exposure.  This  data  division  is  the  only  one  tested  that  clearly  indicates  the  benefit 
of  increased  system  usage.  Using  volunteers  as  the  treatment  group  may  have 
compromised  the  validity  of  the  experiment  somewhat,  but  the  results  may  provide  a 
better  indication  of  what  can  be  expected  in  fleet  application.  The  larger  proportion  of 
officers  using  the  system  in  the  May  class  also  creates  interesting  questions  regarding 
computer  ability. 

D.  ANALYSIS  OF  CENSORED  DATA  SET 

As  mentioned  in  Chapter  IV.B.l,  additional  variables  were  available  for  52 
participants  of  the  study.  The  censored  data  set  is  provided  in  Appendix  D.  Subject  race. 
Armed  Forces  Qualifying  Test  Score  and  Mental  Category,  as  well  as  scores  achieved  on 
the  various  Armed  Services  Vocational  Aptitude  Battery  subtests  were  provided  by  the 
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DMDC  database.  As  such,  this  information  was  included  in  the  analysis  in  order  to 
determine  variable  influence  on  each  subject’s  performance  during  the  Oral  Board 
administered  at  the  conclusion  of  the  Repair  Party  Leader  course. 

1.  Variable  Selection 

Appendix  E  provides  the  results  of  stepwise  regression  on  13  independent 
variables  using  the  “forward  inclusion”  method  provided  by  S-Plus.  As  shown  in 
Appendix  E,  the  variables  were  added  in  the  following  order: 

1.  Race 

2.  Group 

3.  ESWS 

4.  Class 

5.  Pre-Test  Score 

6.  Sea  Duty  Experience  (SeaTime) 

7.  Rank 

8.  Mental  Category  (MenCat) 

9.  AFQT 

10.  Exposure  Time 

11.  Education 

12.  Computer  Aptitude  Index 

13.  Gender 

Figure  20  provides  a  graphical  representation  of  the  impact  the  addition  of  each  variable 
has  on  the  reduction  in  the  residual  sum  of  squares  (numbers  on  the  x-axis  correspond  to 
the  independent  variable  listed  above).  As  shown  in  Figure  20,  the  curve  has  a  steep 
slope  until  the  model  contains  six  or  seven  variables,  and  then  flattens  out,  indicating  that 
the  addition  of  variables  eight  through  13  do  not  provide  a  significant  decrease  in  R2. 
This  result  provides  visual  evidence  that  level  of  education,  computer  aptitude,  gender  or 
time  spent  working  through  scenarios  using  IDCTT  (exposure  time)  did  not  noticeably 
influence  the  subject’s  performance  on  the  Oral  Board. 
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Figure  20.  Variable  Selection  via  Stepwise  Regression. 


2.  Analysis  of  Variance 

Using  die  results  of  the  stepwise  regression,  ANOVA  was  conducted  on  a  model 
employing  the  best  six  variables.  Appendix  F  provides  the  results  of  analysis  of  variance 
of  the  six  variable  model,  including  interaction.  As  shown,  this  analysis  indicated  that 
race,  group  (i.e.  control  or  treatment),  ESWS  qualification,  class  and  pre-test  score  were 
significant  predictors  of  a  student's  oral  board  score.  Rank,  while  one  of  the  variables 
identified  during  the  stepwise  regression  as  important,  is  an  insignificant  predictor  of 
performance  on  the  oral  board  as  indicated  by  this  model  (p-value  =  0.1289).  The 
complicated  interactions  present  between  the  variables  are  difficult  to  understand  or 
interpret;  therefore,  these  interactions  will  be  disregarded  for  this  analysis. 

A  linear  model  was  developed  using  the  results  of  the  ANOVA  to  further  explain 
the  data.  This  model  eliminates  the  contribution  of  rank,  and  renders  the  output  presented 
in  Table  7.  This  model  provides  an  R2  of  0.4175,  indicating  that  42%  of  the  variance  in 
this  data  is  explained  by  these  six  variables.  Of  note,  the  score  obtained  by  the  student  on 
the  pre-test  is  a  poor  predictor  of  how  that  student  will  do  on  the  end  of  course  oral  board. 
Additionally,  the  class  the  student  was  in  is  insignificant  when  examined  in  this  context. 
This  result  will  be  shown  to  be  of  questionable  value  in  later  discussion.  The  linear 
model  also  confirms  that  pre-test  score  is  an  insignificant  predictor  of  oral  board 
performance. 
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Value 

Std.  Error 

t  value 

Pr(>|t|) 

Intercept 

30.9527 

12.9658 

2.3873 

0.0211 

Race 

-7.6349 

2.6019 

-2.9344 

0.0052 

Group 

10.4085 

3.8609 

2.6959 

0.0098 

ESWS 

8.4458 

3.8880 

2.1723 

0.0350 

Class 

4.2186 

2.4236 

1.7406 

0.0884 

Pre.TestScore 

0.2577 

0.1563 

1.6491 

0.1059 

Table  7.  Reduced  (five  variable)  Linear  Model  (Censored  Data). 


Within-group  variability  was  larger  for  the  treatment  group,  suggesting  there  was 
an  uneven  distribution  of  achievement  using  IDCTT  (RPL).  Figure  21  shows  that,  while 
the  median  score  attained  by  the  treatment  group  was  higher,  this  group  also  had  slightly 
higher  variability  (a2=  344.83  for  the  treatment  group,  a2—  161.44  forthe  control  group). 
The  variance  of  the  treatment  and  control  group  mean  oral  board  scores  are  formally 
examined  in  section  C.3  of  this  chapter. 


Figure  21.  Boxplot  of  Control  and  Treatment  Group  Oral  Board  Scores. 
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The  uneven  distribution  of  achievement  for  the  treatment  group  could  be 
attributed  to  any  one  or  all  of  the  variables.  Figure  22  examines  four  of  these  variables: 


Class  1  Class  2  Class  3  Enlisted  Officer 


Figure  22.  Boxplots  of  Treatment  Group  Variability. 

ethnic  group,  advanced  qualifications,  class  in  which  the  subject  was  enrolled,  and 
paygrade.  These  plots  indicate  that  there  was  a  visual  difference  in  performance  within 
the  treatment  group  based  on  ethnic  group,  ESWS  qualification  and,  most  strikingly,  the 
class  the  student  attended.  The  evidence  that  class  is  significant  contradicts  earlier 
findings,  but  makes  sense  when  the  uncensored  data  is  reviewed.  Based  on  the  boxplot  in 
Figure  22,  it  does  not  appear  very  important  whether  the  student  was  an  officer  or 
enlisted.  These  results  generally  support  the  findings  of  the  previously  conducted 
ANOVA. 


3.  Hypothesis  Testing 

The  power  function  developed  for  the  censored  data  set  is  presented  as  Figure  23. 
Since  the  actual  difference  between  the  control  and  treatment  group  mean  oral  board 
scores  was  10.27314,  the  range  of  hypothesized  differences  was  established  at  (-30,  30). 
In  contrast  to  the  previously  shown  power  functions,  the  test  in  this  case  has  power  equal 
to  approximately  0.65  when  the  hypothesized  mean  difference  is  equal  to  ten  percent. 
This  test  is  more  powerful  across  the  range  of  hypothesized  differences,  providing  a 
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Figure  23.  Power  Function  for  Censored  Data  Set. 

power  equal  to  1  at  A  »  15  percent  (where  A  is  the  difference  between  treatment  and 
control  group  mean  oral  board  scores),  as  opposed  to  values  greater  than  20  percent  for 
the  other  data  sets. 

The  issue  remains  to  determine  if  use  of  the  IDCTT  (RPL)  system  enhanced  the 
performance  of  the  students  on  the  oral  board.  Figure  24  provides  some  evidence  that  the 
treatment  is  effective,  but  the  extent  of  the  benefit  of  the  IDCTT  (RPL)  in  enhancing  oral 


Oral  Board  Score  Oral  Board  Score 

(Censored  Data)  (Censored  Data) 

jlL. . 

20  40  CO  80  100 


Oral  Board  Score  Oral  Board  Score 

(Censored  Data)  (Censored  Data) 


0  20  40  60  80  100  120  -2  -1  0  1  2 

x  Quantiles  of  Standard  Normal 


Figure  24.  Diagnostic  Plots  for  Censored  Data  Set 


58 


board  performance  is  unknown.  The  histogram  and  QQ-plot  in  Figure  24  indicate  the 
near  normality  of  the  oral  board  scores  recorded  from  the  censored  data  set.  The  QQ-plot 
indicates,  however,  the  presence  of  positive  skew  and  the  possibility  of  high  outliers. 
Several  transformations  were  attempted,  but  none  produced  results  that  appeared  more 
normal.,  so  the  raw  data  was  retained,  and  was  assumed  to  be  Normally  distributed  for 
analytical  purposes. 

Once  the  normality  of  the  data  had  been  established,  a  test  for  the  equality  of 
sample  variance  was  in  order.  To  determine  whether  the  variance  of  the  control  and 
treatment  group  oral  board  scores  were  the  same,  the  F-test  was  used  to  check  the 
following  hypothesis: 

Ho:  The  variances  of  the  censored  data  set  oral  board  scores  recorded  by  the 

control  and  treatment  group  were  equal. 

versus 

Hi :  The  variances  were  not  equal. 

Under  this  procedure,  an  F-statistic  of  2.1359  was  produced,  along  with  a  p-value  of 
0.0615.  Based  on  a  level  of  significance  equal  to  0.05,  the  null  hypothesis  could  not  be 
rejected.  Therefore,  for  the  purposes  of  this  analysis,  the  variances  in  question  were 
assumed  to  be  equal 

In  order  to  determine  the  relative  performance  of  the  students  in  the  treatment 
group  relative  to  that  of  the  control  group,  the  following  hypotheses  were  established: 

Ho:  The  mean  oral  board  score  recorded  by  the  control  and  treatment  group 

were  equal. 

versus 

Hi:  The  treatment  group  recorded  a  higher  mean  oral  board  score. 

To  test  this  hypothesis,  a  two- sample  t  test  was  conducted.  This  test  indicated  that  at  a 
level  of  significance  of  0.05  and  a  /-statistic  of 2.3434,  there  was  a  difference  between  the 
means  (p-value  =  0.0116)  and  the  null  hypothesis  was  rejected.  In  fret,  the  mean  oral 
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board  score  of  the  control  group  was  48.83,  while  the  treatment  group  recorded  a  mean  of 
59.10.  As  a  result  of  the  use  of  the  IDCTT  (RPL),  the  treatment  group  achieved  a 
learning  benefit  of  21.03%  over  the  control  group,  a  finding  that  is  consistent  with 
improvements  noted  in  other  studies  of  the  effectiveness  of  multimedia  training  systems 
when  used  in  military  applications  (Fletcher,  1991). 

4.  Analysis  of  Results 

Initially,  it  was  thought  that  level  of  education  and  computer  aptitude  would  have 
a  noticeable  impact  on  a  student’s  ability  to  use  the  IDCTT  (RPL),  and,  consequently  on 
his  or  her  performance  during  the  post-treatment  oral  board.  Clearly,  this  was  not  the 
case.  In  feet,  education,  and  computer  aptitude  were  eleventh  and  twelfth  in  importance 
as  shown  by  stepwise  regression.  This  result  was  counterintuitive.  As  one  would  expect, 
however,  being  in  the  treatment  group  was  a  significant  factor  contributing  to  the  higher 
oral  board  scores,  indicating  that  the  students  did  receive  benefit  from  exposure  to  the 
interactive  multimedia  learning  system.  The  class  the  student  was  in  was  also  significant, 
leading  to  the  conclusion  that  either  there  were  differences  in  the  three  learning  groups 
(Le.  the  volunteers  in  the  May  class  performed  better),  or  that  the  conduct  of  the  oral 
board  was  inconsistent  between  classes  convened  in  March  and  April  and  the  class 
convened  in  May.  Qualifications  earned  prior  to  attending  the  RPL  course  also  had  an 
effect  on  the  result.  A  priori,  it  was  thought  that  students  who  were  Enlisted  Surface 
Warfare  Specialists  or  Surface  Warfare-qualified  officers  would  perform  better  on  the 
oral  board,  regardless  of  the  training  track  assigned.  This  was  borne  out  by  the  results  in 
that  ESWS  was  a  significant  factor. 

E.  SUMMARY  OF  RESULTS 

Table  8  provides  summary  information  regarding  the  results  obtained  from  this 
study,  using  the  hypothesis  listed  below: 
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Ho:  The  mean  oral  board  score  recorded  by  the  control  and  treatment  group  were 
equal. 

versus 

Hi:  The  treatment  group  recorded  a  higher  mean  oral  board  score. 


Data 

Significant  Variables 
(ANOVA) 

Result  of 
Hypothesis  Test 

Mean  OBS 

Quantitative 

Learning 

Advantage 

Control 

Treatment 

Complete  Set 

Group,  Class, 

Null  rejected 

48.47 

60.78 

20.26% 

May  97  Class 

Group 

.Null  rejected 

45.06 

80.06 

77.54% 

Censored  Set 

Race,  Group,  ESWS, 

Null  rejected 

48.83 

59.10 

21.03% 

Class,  PreTest 

Table  8.  Results  Summary. 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

The  purpose  of  this  thesis  was  to  determine  whether  using  IDCTT  (RPL)  as  a 
supplemental  learning  tool  would  produce  a  Repair  Party  Leader  School  graduate  that 
was  closer  to  qualification  than  those  students  trained  using  the  traditional  schoolhouse 
method  alone.  As  stated  in  Chapter  V.A.4,  the  following  hypothesis  was  tested  to 
determine  the  impact  use  of  IDCTT  (RPL)  had  on  the  learning  of  students  in  RPL 
School: 


Ho:  The  mean  oral  board  score  recorded  by  the  control  and  treatment  group  were 
equal. 

versus 

Hi:  The  treatment  group  recorded  a  higher  mean  oral  board  score. 

Based  on  the  information  contained  in  Table  8,  it  can  be  concluded  that  use  of 
IDCTT  (RPL)  at  the  Repair  Party  Leader  School  in  San  Diego  did  have  a  positive  impact 
on  the  oral  board  scores  recorded  by  the  treatment  group.  While  the  amount  of  advantage 
gained  by  the  treatment  group  was  fairly  consistent  between  the  complete  data  set  and  the 
members  of  the  censored  set,  the  huge  advantage  gained  by  the  treatment  group  in  the 
May  class  is  noteworthy.  Learning  advantage  may  not  be  a  good  measure  of  progress 
along  the  curve  of  final  qualification,  but  performance  on  the  oral  board  provides  an 
indication  as  to  whether  the  RPL  candidate  possesses  the  basic  knowledge  expected  of  a 
Repair  Party  Leader  aboard  ship. 

As  previously  mentioned  the  treatment  group  for  the  May  class  was  comprised  of 
volunteers.  Review  of  literature  presenting  experiments  of  this  type  did  not  address  the 
effect  motivation  would  have  on  the  results.  Since  the  May  class  performed  substantially 
better  than  the  other  two  classes,  it  would  appear  that  the  motivated  student  is  able  to 
achieve  greater  benefit  by  using  the  system  than  a  possibly  resistant  subject.  Of  course,  it 
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could  be  hypothesized  that  the  motivated  student  would  perform  better  anyway, 
dampening  the  attribution  of  success  to  use  of  IDCTT  (RPL). 

It  was  surprising  that  previously  attained  advanced  qualifications  (ESWS),  rank 
and  education  (including  computer  aptitude)  were  not  found  to  be  significant  factors  in 
this  research.  This  lack  of  significance  would  lead  to  the  belief  that  IDCTT  (RPL)  is  a 
very  user-friendly  system,  one  that  makes  it  easy  for  the  student  to  proceed  through  the 
scenario,  gaining  experience  along  the  way.  Additionally,  it  could  be  stated  that  when 
coupled  with  a  rigorous  training  program  (like  RPL  School),  the  level  of  qualification 
previously  held  by  the  student  is  not  as  important  as  might  have  been  assumed.  This  is  a 
positive  aspect  of  this  interactive  system  in  that  its  employment  in  the  fleet  will  be  to 
train  Sailors  of  all  ranks,  rates  and  levels  of  experience: 

B.  RECOMMENDATIONS  FOR  IDCTT  (RPL)  IMPROVEMENT 

IDCTT  (RPL)  is  a  very  easy  system  to  use,  and  the  computer  on  which  the 
software  was  installed  was  fast  and  capable.  However,  the  limited  number  of  scenarios 
may  accelerate  the  student’s  loss  of  interest  in  conducting  the  number  of  repetitions 
required  to  become  a  proficient  Repair  Party  Leader.  The  following  are 
recommendations  for  IDCTT  (RPL)  improvement. 

1 .  Improve  the  feedback  system. 

Currently,  the  post-scenario  feedback  provided  to  the  student  consists  of  a  brief 
pass/fail  assessment  of  performance,  followed  by  a  review  of  actions  taken  by  the  user. 
This  could  be  embellished  by  designing  the  system  to  reinforce  actions  taken  correctly,  as 
well  as  pointing  out  procedural  errors,  steps  taken  out  of  sequence  and  errors  made  due  to 
lack  of  knowledge  regarding  ship’s  systems.  A  major  portion  of  the  debrief/critique 
presented  at  the  conclusion  of  an  actual  damage  control  drill  addresses  what  the  Repair 
Locker  did  correctly,  as  well  as  presenting  discrepancies.  An  enhancement  of  the 
feedback  system  would  allow  either  supervisory  personnel  or  the  motivated  user  to  tailor 
a  training  package  to  address  shortcomings  discovered  during  the  scenario.  For  greatest 


impact,  IDCTT  (RPL)  could  provide  a  printed  report  of  errors,  the  action  that  should  have 
been  taken,  and  the  reference  for  the  step. 

2.  Simplify  the  process  required  to  locally  develop  scenarios. 

Scenarios  available  today  are  provided  by  die  prime  software  contractor.  Making 
it  possible  for  the  shipboard  damage  control  organization  to  locally  generate  scenarios 
would  be  a  noteworthy  enhancement  in  utility  of  the  system  aboard  ship.  The  ability  to 
build  and  test  scenarios  would  provide  at  least  two  benefits.  First,  the  ship  would  be  able 
to  provide  tailored  training  to  its  Repair  Locker  Leaders.  Second,  the  Damage  Control 
Training  Team  would  be  able  to  prepare  scenarios  for  live  drills,  and  then  test  them  for 
pitfalls  prior  to  actually  placing  ship’s  company  at  General  Quarters  and  executing  the 
drill  plan.  This  would  decrease  time  spent  developing  quality  drills,  and  would  enhance 
the  amount  of  benefit  damage  control  personnel  would  gain  per  hour  spent  at  General 
Quarters.  A  well-designed  drill  makes  a  big  difference  in  the  training  received  by  the 
crew. 


3.  Add  a  tutorial  package  to  the  existing  software. 

Alessi  and  Trollip  (1991)  describe  training  as  a  four-part  process.  In  its  current 
configuration,  IDCTT  (RPL)  does  not  address  the  aspect  of  “presenting  the  information” 
to  the  student.  Addition  of  an  in-depth  tutorial  covering  advanced  damage  control  theory 
and  procedures  is  necessary  in  order  to  increase  the  training  ability  of  the  system. 
Ideally,  students  would  be  able  to  select  a  training  package  covering  a  particular  aspect  of 
damage  control,  and,  after  taking  the  tutorial,  immediately  apply  the  material  in  the 
tutorial  to  a  scenario  selected  from  the  menu.  This  rapid  use  of  knowledge  would  serve 
to  reinforce  what  has  been  learned,  increasing  the  amount  retained  by  the  user. 

4.  Add  an  on-line  library. 

A  noteworthy  shortcoming  of  IDCTT  (RPL)  in  its  current  configuration  is  the 
lack  of  an  on-line  library  of  damage  control  publications.  While  many  actions  required 
of  the  RPL  should  be  committed  to  memory,  many  others  allow  for  thought  and  review  of 
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information.  In  a  shipboard  repair  locker,  the  RPL  has  access  to  many  sources  of 
information,  including  a  large  library  of  pertinent  damage  control  publications.  By  being 
able  to  access  these  during  a  scenario,  the  training  environment  would  be  more  realistic, 
and  would  also  familiarize  the  RPL  with  reference  assets  contained  in  his  or  her  repair 
locker. 


5.  Provide  scenarios  for  other  ship  classes. 

Currently,  IDCTT  (RPL)  scenarios  exist  only  for  DDG-51  class  ships.  While  this 
class  of  ship  comprises  a  significant  portion  of  the  Navy’s  surface  combatants,  there  are 
numerous  other  classes  that  are  underrepresented.  In  order  to  enhance  the  fleet-wide 
utility  of  IDCTT  (RPL),  scenarios  based  aboard  cruisers,  guided  missile  frigates, 
amphibious  and  combat  logistics  force  ships  should  be  developed  as  soon  as  possible. 

C.  RECOMMENDATIONS  FOR  FURTHER  RESEARCH 

This  study  provides  evidence  that  using  IDCTT  (RPL)  as  an  educational  augment 
to  the  traditional  Repair  Party  Leader  School  curriculum  would  be  of  benefit  to  the 
students  enrolled  in  the  course.  There  are,  however,  several  additional  areas  of  research 
that  could  be  pursued  in  order  to  validate  the  findings  of  this  study. 

1.  Improve  the  experimental  design. 

In  order  to  provide  a  better  measure  of  effectiveness  for  IDCTT  (RPL)  as  tested  at 
the  Repair  Party  Leader  School,  the  pretest  should  take  the  form  of  an  oral  board 
comprised  of  questions  similar  to  those  asked  at  the  end  of  the  course.  This  change 
would  provide  a  better  indication  of  entrance  level  of  knowledge  pertinent  to  the  type  of 
learning  that  would  be  expected  to  be  delivered  by  IDCTT  (RPL).  As  conducted,  it  was 
difficult  to  relate  pre-test  scores  and  oral  board  results. 
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2.  Other  research  opportunities. 

Aside  from  ensuring  that  the  Repair  Party  Leader  is  fully  competent  at  the 
completion  of  the  qualification  process,  the  most  important  factor  is  the  amount  of  time 
required  to  achieve  shipboard  RPL  certification.  Other  studies,  including  Army- 
sponsored  research  regarding  the  training  of  communications  specialists  (Winkler  and 
Polich,  1991),  report  that  interactive  training  systems  significantly  reduce  the  time 
required  for  a  trainee  to  develop  a  skill.  If  students  attending  RPL  School  were 
identified,  trained  with  IDCTT  (RPL)  as  a  supplement  to  traditional  training,  and  tracked 
to  completion  of  qualification  aboard  ship,  the  time  required  to  complete  this  process 
could  be  compared  with  those  not  receiving  IDCTT  (RPL)  training.  This  information 
could  be  used  as  support  for  a  cost-benefit  study  on  which  to  base  procurement  and 
system  fielding  decisions.  Finally,  an  additional  study  could  be  commissioned  to 
determine  whether  the  effect  was  due  to  instruction  or  could  be  attributed  directly  to  use 
of  the  IDCTT  (RPL).  While  the  result  of  this  research  is  encouraging  with  respect  to  the 
prospect  of  using  interactive  methods  to  train  Repair  Party  Leaders,  it  is  still  not  clear 
what  the  difference  will  be  when  used  in  fleet  applications.  A  study  conducted  aboard 
ship,  comparing  the  actual  performance  of  school  graduate  RPL’s  with  those  trained 
using  only  IDCTT  (RPL)  would  provide  additional  information  relative  to  the  system’s 
effectiveness. 


67 


APPENDIX  A. 


IDCTT  (RPL)  TEACHING  POINTS 


Dr.  Eric  Allely  of  Tekamah  Company  provided  these  teaching  points.  Tekamah 
develops  the  source  code  for  all  scenarios  used  in  IDCTT  (RPL).  The  listed  teaching 
points  were  current  as  of  March,  1997. 

Teaching  Point  Pitfall:  action  not  taken 

1 .  Report  Repair  3  Manned  and  Ready 

The  Manned  and  Ready  report  signifies  that  personnel  are  available  to  combat 
casualties  and  are  prepared  with  proper  equipment,  personnel  protection  and 
communications.  It  is  imperative  that  the  senior  person  in  the  Repair  Station 
provides  the  report  to  DC  Central  as  soon  as  personnel  are  ready  to  commence 
fighting  the  casualty.  (NWP62-1D) 

Manned  and  Ready  time  requirements  are  contained  in  FXP-4  exercise  MOB-D- 
3-SF.  For  a  DD/DDG  class  ship  the  time  standard  for  Manned  and  Ready  is  5 
minutes. 

2.  Zebra  set. 

Material  Condition  zebra  provides  the  greatest  degree  of  subdivision  and 
watertight  integrity  to  the  ship.  It  is  the  maximum  state  of  readiness. 

In  FXP-4  exercise  MOB-D-1 1-SF,  the  time  standard  for  setting  material  condition 
zebra  for  a  DD/  DDG  class  ship  is  5  minutes 

3.  Modified  Zebra 

Certain  situation  may  require  the  CO  to  order  material  condition  modified  zebra 
set.  Modified  material  condition  zebra  will  give  a  higher  survivability  stance  than 
condition  yoke,  while  modified  zebra  is  less  restrictive  and  will  allow  the 
accomplishment  of  certain  operational  requirements.  (DC  PQS) 

4.  Changing  material  condition  (Y,  W,  circle  W) 

Changes  in  the  ship's  material  condition  require  Commanding  Officer  approval 
(NWP  3-20.31) 

5.  Investigation 

Investigation  teams  should  be  rapidly  dispatched.  Any  delays  in  investigating 
damage  may  result  in  the  further  progressive  spread  of  damage.  Investigation 
shall  be  thorough,  conducted  with  caution,  clearly  and  quickly  reported  and 
investigation  is  to  be  repeated  (DC  PQS) 
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6. 


Flooding  Boundaries 


The  setting  of  flooding  boundaries  should  be  an  immediate  action  when  flooding 
is  reported  to  limit  progressive  flooding  into  adjacent  compartments. 

Flooding  boundaries  should  be  set  at  the  main  transverse  watertight  bulkheads 
closest  to  the  flooding.  (DC  PQS) 

7.  Fire  Main  valve  isolation 

Rapid  firemain  valve  isolation  will  reduce  flooding  from  ruptured  piping. 
Firemain  isolation  should  be  coordinated  with  DC  Central  to  prevent  disruption  to 
vital  systems  served  by  the  firemain. 

8.  Hole  patched 

Failure  to  patch  a  hole  in  the  bulkhead  will  result  in  progressive  flooding  into 
adjacent  compartments. 

9.  Progressive  flooding 

10.  Shore  Bulkhead 

Failure  to  shore  a  panting  bulkhead  may  result  in  the  loss  of  the  bulkhead  and 
damage  to  the  adjacent  compartment. 

Indications  of  the  need  for  shoring  include  deep  bulges  in  plating,  bowed  frames 
and  stanchions,  loose  rivets,  cracked  seams,  and  panting  bulkheads.  Panting  is  a 
dangerous  condition.  It  results  in  metal  fatigue.  Which  eventually  causes  cracking 
and  splitting.  (Repair  Party  Manual) 

When  in  doubt  always  shore!  (Repair  Party  Manual) 

1 1 .  Request  additional  personnel  for  Flooding 

Requests  for  additional  personnel  should  be  coordinated  via  Damage  Control 
Central.  A  failure  to  keep  DC  Central  informed  and  to  coordinate  limited  Damage 
Control  resources  may  result  in  repair  teams  being  overwhelmed  by  cascading 
casualties. 

12.  Request  route 

Routing  of  personnel  transiting  the  Repair  Station  area  should  be  coordinated  with 
DC  Central 

13.  Stability  is  Critical 
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The  "Basic  Thumbrules  of  Critical  Stability"  are  contained  in  the  CNSL/CNSP 
Repair  Party  Manual.  All  Repair  Party  Leaders  should  be  familiar  with  basic 
stability  concepts.  In  some  circumstances,  particularly  when  inport,  an  RPL  may 
be  the  only  person  onboard  with  training  in  stability  principles. 

Stability  may  be  is  critical  if  any  of  the  following  conditions  exist  following 
damage: 

Small  or  negative  metacentric  height 
exceeding  floodable  length 
excessive  list 
heavy  winds  and  seas 

Small  or  NegativeMetacentric  Height 

When  stability  is  critical  due  to  small  or  negative  metacentric  height: 

a.  The  ship  is  logy  with  a  slow  erratic  roll  period  and  a  tendency  to  hang  at 
the  end  of  the  roll 

b.  The  ship  has  a  tendency  to  list  at  the  same  angle  to  either  side. 

c.  The  ship  has  a  list  which  cannot  be  accounted  for  by  off-center  weight. 
(Repair  Party  Manual) 

Exceeding  Floodable  Length 

When  Stability  is  critical  due  to  exceeding  floodable  length,  the  extent  of  flooding 
approaches  or  exceeds  the  maximum  amount  of  flooding  tabulated  in  the  ships 
Damage  Control  Book,  section  II  (a)  (Repair  Party  Manual) 

Excessive  List 

When  Stability  is  critical  due  to  excessive  list,  the  ship  lists  to  a  static  heel  of  15 
degrees  or  more.  (Repair  Party  Manual) 

Heavy  Winds  and  Seas 

When  Stability  is  critical  due  to  heavy  winds  and  seas.  Heavy  winds  and  rough 
seas  are  prevailing  or  are  anticipated.  (Repair  Party  Manual) 

14.  Counterflooding  or  Ballasting 

Counterflooding  or  Ballasting  requires  the  approval  of  the  Commanding  Officer 
(NWP  3-20.31) 

15.  Jettisoning 

Jettisoning  requires  the  approval  of  the  Commanding  Officer  (NWP  3-20.31) 

16.  Starting/Stopping  Firepumps 

Firemain  management  requires  close  coordination  with  DC  Central.  The  starting 
and  stopping  of  fire  pumps  should  be  ordered  by  DC  Central. 
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1 7.  Dewater  using  installed  Eductor  (No  FM) 

Installed  eductors  require  proper  firemain  pressure  to  operate.  Improper 
alignment  of  the  ship's  firemain  system  when  using  eductors  may  result  in 
accidental  flooding  rather  than  dewatering  of  a  compartment. 

18.  Fire  Boundaries 

Any  physical  harrier  can  be  a  fire  boundary.  Ideally  fire  boundaries  are  the 
bulkheads,  deck,  and  overhead  surrounding  the  fire.  Secondary  boundaries  are 
generally  set  at  watertight  subdivisions  or  airtight  boundaries.  (NWP  62- ID) 

19.  Smoke  Boundaries 

Smoke  boundaries  are  set  to  contain  smoke  within  a  fixed  area  and  prevent  the 
spread  of  smoke  either  horizontally  or  vertically.  (NWP  62- ID) 

20.  Failure  to  Request  additional  personnel  for  F/F 

Rapidly  manning  fire  teams  may  be  difficult  if  other  Damage  Control  procedures 
are  already  in  progress  and  repair  personnel  have  been  assigned  to  other  teams. 
Emerging  fire  team  requirements  should  be  coordinated  with  DC  Central  to 
ensure  the  fastest  response  and  prevent  fires  from  rapidly  spreading. 

2 1 .  Electrical  Isolation 

The  extent  of  compartment  electrical  isolation  and  the  securing  of  lighting  is 
typically  determined  by  the  scene  leader.(  NSTM  555) 

22.  Mechanical  Isolation 

When  a  space  is  abandoned  due  to  fire,  the  space  should  be  mechanically  and 
electrically  isolated  to  the  greatest  extent  possible.  A  CO  may  choose  not  to 
isolate  such  spaces  if  they  are  essential  to  the  safety,  mobility,  or  fighting 
capability  of  the  ship  (NSTM  555). 

23.  Securing  of  ventilation 

The  decision  to  secure  ventilation  systems  is  made  on  scene. (NWP  62- ID) 

24.  Magazine  Sprinkler  activation 

Activation  of  magazine  sprinklers  requires  the  approval  of  the  Commanding 
Officer.  (NWP  3-20.31) 

25.  Magazine  sprinklers  activation,  no  FM 
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Firemain  management  requires  attention  to  vital  systems  served  by  the  firemain 
including  magazine  sprinkler  systems. 

26.  Movement  of  Ammunition 

The  movement  of  ammunition  onboard  the  ship  requires  approval  from  the 
Commanding  Officer.  Ammunition  movement  should  be  conducted  by  trained 
Combat  Systems  personnel  and  should  be  coordinated  via  DC  Central  and 
Combat  Systems  Maintenance  Central. 

27.  Halon  activation 

Halon  activation  may  prove  ineffective  if  the  compartment  has  been  sufficiently 
damaged  to  prevent  Halon  from  reaching  the  proper  concentration  in  the  space  to 
extinguish  the  fire. 

28.  Jettison  of  ammunition 

The  jettisoning  of  objects  over  the  side  requires  Commanding  Officer  approval. 
(NWP  3-20.31)  The  movement  of  ammunition  (including  moving  it  over  the 
side)  also  requires  Commanding  Officer  approval. 

29.  Jettison  of  Gas  Cylinders 

The  jettisoning  of  objects  over  the  side  requires  CO  approval.  (NWP  3-20.31) 
Evacuation  of  Repair  3  (smoke) 

The  evacuation  of  personnel  within  the  Repair  Station  area  to  the  weatherdecks 
may  be  required  when  smoke  boundaries  are  not  correctly  ordered  and  set. 

30.  Direct  Attack 

In  a  direct  attack,  firefighters  advance  into  the  immediate  fire  area  and  apply  the 
extinguishing  agent  directly  onto  the  seat  of  the  fire.  (NSTM  555) 

31.  Indirect  Attack 

An  indirect  attack  is  the  application  of  water  fog  into  the  fire  space  through  an 
existing  access  or  through  a  hole  cut  in  a  bulkhead  or  overhead  when  heat  or  other 
conditions  deny  access  to  the  space,  an  indirect  attack  may  improve  condition 
conditions  to  permit  reentry  for  a  direct  attack..  (NSTM  555) 

32.  Active  Desmoking 
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Active  desmoking  is  removing  smoke  and  heat  from  the  smoke  control  zone  prior 
to  extinguishing  the  fire,  to  aid  firefighting  efforts  and  reduce  smoke  spread  in  the 
ship.  (NSTM  555) 

33.  Overhaul  of  Fire 

Overhaul  of  a  fire  is  an  examination  and  cleanup  operation.  It  includes  finding 
and  extinguishing  hidden  fires  and  determining  whether  the  fire  has  extended  to 
other  areas  of  the  ship.  (NSTM  555) 

34.  Desmoke 

After  a  compartment  fire  has  been  extinguished,  combustible  gases  may  be 
present.  The  goal  of  desmoking  is  to  replace  95%  of  the  smoke  laden  air  with 
fresh  air.  This  will  require  approximately  4  complete  space  volume  changes  in  the 
compartment.  (NSTM  555) 

35.  Atmospheric  Testing 

All  spaces  should  be  desmoked  before  atmospheric  testing  is  started  because 
oxygen  sensors  do  not  operate  reliably  if  the  sensor  is  exposed  to  excessive 
moisture  or  is  in  contact  with  postfire  atmospheric  particulates. 

36.  Request  Personnel  Casualty  Transport  Route 

FXP-4  exercise  FSO-M-2-SF  states  that  a  safe  route  for  stretcher-bearers  should 
be  received  from  DC  Central.  The  FXP-4  time  standard  for  stretcher-  bearers  to 
arrive  on  scene  is  within  10  minutes  of  the  casualty  report. 

37.  Manning  Flight  Quarters 

Due  to  damage  control  actions  already  in  progress  in  the  Repair  3  area,  manning 
the  Aviation  Crash  and  Salvage  Team  exclusively  from  Repair  3  may  result  in 
Repair  team  manning  shortfalls.  As  RPL  you  should  inform  DC  Central  of 
anticipated  problems  and  request  additional  personnel  when  necessary. 

38.  Hazardous  Material  Incident  Response 

Any  material  that  because  of  its  quantity,  concentration,  or  physical  or  chemical 
characteristics  may  pose  a  substantial  hazard  to  human  health  or  the  environment 
when  released  or  spilled.  (DC  PQS) 

39.  Casualty  Power 

The  engineering  officer  will  designate  the  switchboard  and  riser  to  be  used  as  a 
source  of  casualty  power.  (Repair  Party  Manual) 
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The  casualty  power  route  should  be  requested  from  DC  Central. 

Permission  to  rig  casualty  power  should  be  requested  from  DC  Central. 

40.  MOPP  Levels 

Changes  to  the  MOPP  level  should  be  directed  by  DC  Central  and  the 
Commanding  Officer. 

41.  Counter  Measure  Washdown  System 

Activation  of  the  counter  measure  washdown  system  (CMWDS)  requires  the 
approval  of  the  ships  Commanding  Officer.  (NWP  62- ID) 

42.  Dispatching  External  CBR  Monitors 

Dispatching  external  CBR  monitors  outside  the  skin  of  the  ship  when  in  a  CBR 
environment  requires  the  Commanding  Officer's  approval  (Repair  Party  Manual) 

43.  Dispatching  External  CBR  Decontamination  Teams 

Dispatching  External  CBR  Decontamination  Teams  outside  the  skin  of  the  ship 
when  in  a  CBR  environment  requires  the  Commanding  Officer's  approval  (Repair 
Party  Manual) 
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APPENDIX  B  STUDENT  ENTRANCE  DATA  SHEET  AND  PRE-TEST 


This  appendix  provides  the  pre-test  administered  to  students  in  the  March,  April 
and  May  1997  Repair  Party  Leader  School  at  Fleet  Training  Center,  San  Diego  California. 
The  first  page  of  this  examination  provided  the  vehicle  for  collection  of  demographic  data 
on  each  participating  student. 


REPAIR  PARTY  LEADER  COURSE  PRE-TEST 


INSTRUCTIONS  FOR  TAKING  TEST 
READ  CAREFULLY 

1.  Purpose  of  the  Test.  This  pre-test  is  given  in  support  of  research  that  is  being  done 
regarding  the  possible  incorporation  of  interactive  courseware  in  the  training  process  at 
this  school.  Your  participation  in  this  study  is  purely  voluntary,  but  you  may  have  fun  and 
really  leam  something!  This  test  is  given  in  order  to  determine  the  baseline  level  of 
knowledge  each  student  has  upon  enrolling  in  the  course.  Your  score  on  this  exam  will 
have  no  impact  on  the  remainder  of  the  course.  The  questions  for  this  test  will  be  multiple 
choice,  true-false,  and  fill-in.  Circle  the  correct  answer  or  fill  in  the  blank  as  required. 

2.  Quickly  fill  in  the  information  requested  on  the  attached  information  sheet. 

3.  If  you  need  assistance  during  the  test,  raise  your  hand.  One  of  the  instructors  will 
assist  you. 

4.  Examples 

A.  MULTIPLE  CHOICE  AND  FILL-IN 

Each  question  has  four  choices;  a,  b,  c,  and  d.  Read  all  four  choices  carefully.  Only 
one  answer  will  be  correct. 

B.  TRUE-FALSE 

If  any  part  of  the  statement  is  false,  the  whole  item  is  to  be  marked  false  on  the  test. 
True  statements  are  to  be  marked  by  circling  ‘True’  on  the  test  sheet.  For  false 
statements,  circle  ‘False’. 

5.  20  minutes  is  allotted  for  this  test.  When  you  have  completed  the  test,  remain  quiet  at 
your  seat  until  told  to  turn  in  your  test. 

6.  CAUTION 

If  you  desire  to  change  an  answer,  erase  your  first  choice  completely,  then  select  your 
new  choice.  ANY  CHOICE  HAVING  TWO  OR  MORE  ANSWERS  SELECTED  IS 
AUTOMATICALLY  CONSIDERED  WRONG.  ANSWER  EVERY  QUESTION.  IF 
YOU  UNDERSTAND  THE  INSTRUCTIONS,  WAIT  FOR  THE  SIGNAL,  AND  THEN 
BEGIN  THE  TEST. 
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STUDENT  INFORMATION  SHEET 


REPAIR  PARTY  LEADER  COURSE 
Date _ 1997 

Name _ Rank/Rate _  ESWS  Y  N 

(circle  one) 


SSN 


Command _ 

(name  and  hull  number) 


Date  Reported  to  Present  Command  (Month/Y ear) _ 

How  much  total  sea  time  have  you  accumulated? _ years _ months 

Where  do  you  go  now  for  GO?  _ 

What  is  your  job  there? _ 

Have  you  ever  attended 

General  Shipboard  Firefighting  School  Y  N 

Advanced  Shipboard  Firefighting  School  Y  N 

DCPO  School  Y  N 

Buttercup  Wet  Trainer  Y  N 

1 .  Fires  on  ships  underway  that  have  no  at-sea  fire  party  will  normally  be  handled  by  the 

a.  duty  section. 

b.  in-port  fire  party. 

c.  repair  party  personnel 

d.  Naval  Station  fire  protection  personnel.  1.1.1 

2.  The  Repair  Party  Leader's  duties  and  responsibilities  include 

a.  establishing  maximum  permissible  exposure  (MPE). 

b.  ordering  the  activation/deactivation  of  the  water  wash  down  system  (WWDS). 

c.  ordering  the  flooding  of  magazines. 

d.  assigning  repair  party  personnel  to  teams.  1.1.2 

3.  Part  II  of  the  Ship's  Damage  Control  book  contains 


a.  general  information. 

b.  damage  control  systems. 
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c.  miscellaneous  systems. 

d.  electrical  systems. 


1.2.1 


4.  Heavy  lines  on  isometric  damage  control  diagrams  indicate 
boundaries. 


a.  water-tight  and  oil-tight 

b.  air-tight 

c.  fire  retardant  and  flame-tight 

d.  non-tight  1.2.1 

5.  A  DC  training  simulation  of  black  cloth  taped  over  an  item  would  indicate 

a.  black  smoke. 

b.  spilled  oil. 

c.  charring. 

d.  class  "B"  fire.  1.3. 1.1 


For  items  6  through  9,  match  the  advantages/limitations  in  Column  A  to  the  DC 
communications  systems  in  Column  B  by  drawing  a  line  between  the  two. 


COLUMN  A 

6.  A  flexible  and  survivable  means  of 
rapid  communications  but  can  not  be 
used  in  a  weapons  handling  incident/ 
accident. 

7.  Requires  no  external  source  of  power. 

8.  Provides  a  fast  and  dependable  two 
way  transmission  between  DCC  and  each 
system  repair  station. 

9.  Not  rugged,  may  go  out  of  commission 
early  in  action,  (requires  electrical  power). 


COLUMN  B 

a.  Intercommunication  system 

b.  DC  WIFCOM. 

c.  Ship's  service  telephones  (J-circuit) 

d.  Sound-powered  telephone 


10.  Select  the  priority  use  of  communications  from  DCC  to  the  repair  lockers. 


a.  3, 1,2  and  4.  1. 

b.  1, 3,  2  and  4..  2. 

c.  3, 2,  4  and  1  3. 

d.  1,  3,  4  and  2  4. 


DC  Wifcom 
MC  Circuit. 

Sound  Powered  Telephones. 

Ship's  Service  Telephone,  (J-circuit). 


80 


11.  Isometric  DC  Diagrams,  Status  Boards,  and  Logs 


a.  are  all  unclassified  documents. 

b.  require  constant  updating  and  close  attention. 

c.  seldom  require  updating. 

d.  are  only  located  on  the  bridge  and  in  DCC. 

12.  A  proper  wedge  has  a  length  of _  times  its  butt  thickness. 

a.  3 

b.  6 

c.  8 

d.  10 

13.  Select  the  first  step  for  by-passing  a  rupture  in  the  firemain. 

a.  Rig  one  2  1/2  inch  hose  from  the  two  nearest  pressurized  sections  beyond  the 
rupture. 

b.  Remove  the  wye-gate  from  the  required  fireplugs. 

c.  Isolate  the  ruptured  section. 

d.  Shut  down  the  fireplug  nearest  the  rupture. 

14.  The  maximum  length  of  a  shore  can  not  be  more  than _ times  it's  minimum 

thickness. 

a.  10 

b.  20 

c.  30 

d.  40 

15.  An  electric  submersible  pump  will  discharge  _  GPM  with  a  70  foot  static  head. 

a.  140 

b.  160 

c.  200 

d.  250 

16.  The  permanent  angle  of  heel  taken  by  a  ship  due  to  flooding  is  called _ . 

a.  roll 

b.  trim 

c.  pitch 

d.  list 
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17.  The  secondary  drainage  system _ 

a.  drains  spaces  forward  and  aft  of  the  main  machinery  compartments. 

b.  has  larger  piping  than  the  main  drainage  system. 

c.  is  connected  to  and  uses  the  same  pumps  and  eductors  as  the  main  drainage 
system. 

d.  drains  the  main  machinery  compartments. 

18.  The  firemain  provides  operating  pressure  to  eductors  in  the _ systems. 

a.  main  and  secondary  drainage 

b.  supply  and  exhaust 

c.  primary  and  alternate 

d.  auxiliary  firemain 

19.  For  desmoking  after  a  fire,  installed  or  fixed  ventilation  systems  can 

a.  never  be  used. 

b.  be  used  with  the  repair  party  leaders  permission. 

c.  be  used  with  restrictions. 

d.  be  used  if  the  exhaust  discharges  into  a  different  space. 

20.  The  Ram  Fan  2000  is  driven  by 

a.  LP  air 

b.  Water  under  pressure 

c.  110  VAC 

d.  HP  air 

21.  Extra  casualty  power  cables  are  stored  in  DC  repair  lockers  in  lengths  of _ feet. 

a.  100 

b.  150 

c.  200 

d.  250 

22.  The  function  of  the  Casualty  Power  System  is  to _ . 

a.  provide  power  to  all  weapon  systems. 

b.  supply  power  to  all  of  the  ships  electrical  equipment. 

c.  make  temporary  power  connections  to  vital  systems/equipment. 

d.  splice  and  repair  permanently  damaged  electrical  components. 
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23.  What  is  the  correct  sequence  to  be  followed  when  rigging  casualty  power  cable? 


a.  Rig  power  source  to  load 

b.  Rig  from  load  to  power  source 

c.  No  preferred  sequence 

d.  From  switchboard  to  power  panel 


24.  An  itemized  list  of  all  damage  control  equipment  located  in  a  compartment  is  called 
a/an  _ . 


a.  Damage  Control  Book. 

b.  Closure  Log. 

c.  Compartment  Check-Off  List,  (CCOL). 

d.  EOSS. 


MATCH  THE  COMPLETED  DAMAGE  CONTROL  MESSAGE  TO  THE 
CORRESPONDING  LETTER  CODE  THAT  CORRECTLY  IDENTIFIES  THE 
MESSAGE. 


25.  Message  No.  1 

a.  Un-jammed  WTH  2-35-2  in  compartment  2-23-0-L. 

b.  Un-jammed  quick  acting  scuttle  2-35-2  in  compartment  2-23-0-L. 

c.  Jammed  WTH  2-35-2  in  compartment  2-23-0-L. 

d.  Un-jammed  quick  acting  WTH  2-35-2  in  compartment  2-23-0-L. 

26.  Message  No.  2 

a.  Electrical  power  lost  in  compartment  3-23-0-E. 

b.  Secured  communications  to  compartment  3-23-0-E. 

c.  Electrical  power  ordered  secured  to  compartment  3-23-0-E. 

d.  Restored  communications  to  compartment  3-23-0-E. 

27.  Message  No.  3 

a.  Minus  oxygen,  plus  toxic  gases  in  compartment  3-23-0-E. 

b.  Plus  oxygen,  plus  explosive  gases  and  plus  toxic  gases  in  compartment  3-23-0-E. 

c.  Plus  oxygen,  minus  explosive  gases  and  minus  toxic  gases  in  compartment 
3-23-0-E. 

d.  Plus  oxygen,  minus  toxic  gases  in  compartment  3-23-0-E. 

28.  Message  No.  4 

a.  Watertight  hatch  3-36-2  is  shored  and  shoring  watch  is  set  in  compartment  3-23- 
0-E. 
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b.  Watertight  door  3-36-2  is  reported  shored  in  compartment  3-23-0-E. 

c.  Watertight  hatch  3-36-2  is  sprung  and  shoring  watch  is  set  in  compartment  3-23- 
0-E. 

d.  Watertight  hatch  3-36-2  in  compartment  3-23-0-E  is  reported  damaged. 

29.  Message  No.  5 

a.  Jammed  watertight  hatch  2-36-2  in  compartment  2-24-0-L. 

b.  Un-jammed  WTH  2-35-2  in  compartment  2-23-0-L. 

c.  Jammed  watertight  hatch  2-35-2  in  compartment  2-23-0-L. 

d.  Un-jammed  watertight  door  2-36-2  in  compartment  2-24-0-L. 


30.  Message  No.  6 

a.  Ruptured  firemain  riser  reported  in  compartment  2-15-1-L. 

b.  Ruptured  firemain  riser  is  repaired  in  compartment  2-15-1-L. 

c.  Desmoking  completed  in  compartment  2-15-1-L. 

d.  Restored  flooding  boundaries  in  compartment  2- 1 5-  1-L. 
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NWP  3  -  20 . 31 ,  Fi  gure  4 . 3,  Sheet  9 
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Message  Nr  2 


NWP  3  -  20 . 31,  Fi  gure  4 . 3,  Sheet  13 
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APPENDIX  C  ORAL  BOARD  QUESTIONS  AND  PROCTOR  INSTRUCTIONS 

This  appendix  provides  the  instructions  given  to  proctors  administering  the  end-of- 
course  oral  boards,  as  well  as  questions  to  he  asked  verbatim.  For  most  of  the  oral  board 
questions,  the  applicable  line  item  from  NAVEDTRA  43119-G,  Watchstation  318  (Repair 
Party  Leader)  has  been  identified.  The  question  is  designed  to  determine  the  individual’s 
knowledge  regarding  that  particular  line  item  of  the  PQS. 


Instructions  to  Oral  Board  Proctor 

The  purpose  of  this  oral  board  is  to  provide  a  post-test  to  students  of  the  Repair  Party 
Course  that  may  assist  in  determining  the  value  of  the  IDCTT  (RPL)  as  a  system  with 
which  to  augment  damage  control  training  delivered  in  the  schoolhouse.  The  following 
instructions  must  be  followed  explicitly  in  order  to  ensure  quality  and  validity  of  data 
collected. 

Read  the  following  paragraph  to  the  student. 

“The  purpose  of  this  oral  board  is  to  gather  data  that  will  help  in  determining  the  value  of 
the  interactive  courseware  that  has  been  installed  here  at  FTC.  This  board  will  have  no 
impact  on  your  final  grade  here  at  the  school.  I  will  now  go  over  the  method  I  will  use  for 
conducting  this  board. 

1.  You  are  to  relax... remember,  this  board  doesn’t  count  toward  your  score  in 
the  course. 

2.  Do  your  best  to  answer  each  question.  Think  about  your  answer  before  giving 
it. 

3.  This  board  will  last  about  15  minutes.  I  will  ask  you  a  question  every  minute, 
on  the  minute.  If  you  are  still  answering  the  previous  question,  I  will  stop  you 
and  we  will  move  to  the  next  question. 

4.  During  the  board,  I  will  answer  no  questions  and  will  not  provide  any 
information  other  than  the  question. 

Before  we  begin,  do  you  have  any  questions?  If  not,  we’ll  start  the  board.” 

Reminders  for  the  Proctor 

1.  Do  not  ask  the  student  if  he/she  was  in  the  group  that  used  IDCTT. 
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2.  Do  not  let  the  student  see  you  writing  your  observations  (i.e.  don’t  jump  on  a 
response  and  make  a  production  out  of  recording  it) 

3.  Ask  the  questions  at  exactly  1  minute  intervals.  If  the  student  answers  the 
previous  question  early,  wait  until  1  minute  has  elapsed,  then  ask  the  next 
question.  If  the  student  is  in  the  middle  of  an  answer  at  the  end  of  the  1  minute 
period,  stop  him/her  and  ask  the  next  question. 

4.  Record  the  student’s  correct  responses  by  placing  checks  in  the  blocks 
corresponding  to  the  response.  You  have  the  authority  to  interpret  the  answer 
given  in  order  to  decide  if  it  is  correct  (i.e.  it  is  understood  that  the  student  will 
probably  not  quote  the  reference  verbatim). 

5.  Do  not  enhance/embellish  the  question.  These  questions  are  scripted  and  must 
be  read  verbatim  to  each  student. 

6.  Use  a  new  sheet  for  each  student.  Ensure  you  record  the  student’s  SSN  on  the 
sheet  prior  to  starting  the  board. 

7.  It  is  most  critical  that  the  boards  be  consistent  from  one  student  to  the 
next. 
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Repair  Party  Leader 
Oral  Board  Questions 

Student  Name _  Board  Date 

SSN  _ 


NAVEDTRA  43119-G  Topic  Task 

3 1 8.2. 1 9  Maintain  communications  with  DC  Say  steps  of  procedure 

Central  and  other  Repair  Stations. 

Question.  You  are  Repair  II  Locker  Leader.  By  ship’s  instruction,  your  locker  is  next  in 
succession  for  control  if  something  happens  to  DC  Central.  How  will  you  ensure  good 
communications  and  control  of  the  ship-wide  DC  situation? 

Answer.  (Reference  NSTM  079V2,  079.38. 173) 

1 .  _ Conduct  periodic  communications  checks  with  DCC. 

2.  _ Maintain  a  plot  of  damage  in  afl  areas  of  the  ship. 

3.  _ Take  positive  control  if  DC  Central  is  out  of  action. 


NAVEDTRA  43119-G  Topic  Task 

318.2.12  Coordinate  Repair  Party  efforts  to  Say  steps  ofprocedure 

combat/control  flooding. 

Question.  List  the  actions  the  Repair  Party  Leader  would  take  if  flooding  were  reported 
in  his/her  area  of  responsibility. 

Answer.  (Reference  CNSLINST  3541.1C/CNSPINST  3541.4B) 

1 .  _ Establish  flooding  boundaries. 

2.  _ Electrically  isolate  the  affected  space. 

3.  _ Mechanically  isolate  the  affected  space. 

4.  _ Prioritize  dewatering  efforts. 

5.  _ Evaluate  the  stability  situation. 


Question.  What  does  this  message  tell  you  as  Repair  Party  Leader?  (Show  Message 
Blank  Nr  1) 

Answer.  (Reference  NWP  3-20.31,  Figure  4.3,  sheet  9) 

_ Message  Nr  1  says  electrical  power  isolated  in  compartment  3-381-1-Q 
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Question.  What  are  the  minimum  responsibilities  of  the  Repair  Party  Leader  in  the  event 
of  fire  in  his/her  area  of  responsibility. 

Answer.  (Reference  NWP  3-20.3 1,  Chapter  9.2. 1.1) 

1 .  _ Ensure  boundaries  are  established. 

2.  _ Coordinate  efforts  to  prevent  vertical  fire  spread. 

3.  _ Provide  logistic  support  to  the  scene  (personnel  and 

equipment) 

4.  _ Maintain  an  accurate  plot. 

5.  _ Make  required  status  reports. 


Question.  List  all  the  damage  control  actions  for  which  you  would  need  to  get 
permission  prior  to  doing. 

Answer.  (Reference  Repair  Party  Manual,  Chapter  1,  Art  102,  NWP  3-20.31, 
Art  1.5.8) 

1 .  _ Ballasting/deballasting 

2.  _ Counter  flooding 

3.  _ Changes  to  established  material  condition  of  readiness 

4.  _ Flooding  weapons  magazines 

5.  _ Desmoking  using  installed  ventilation 


Question.  What  does  this  message  tell  you  as  Repair  Party  Leader?  (Show  Message 
Blank  Nr  21 

Answer.  (Reference  NWP  3-20.31,  Figure  4.3,  sheet  13) 

_ Message  Nr  2  says  a  4  inch  hole,  2  feet  off  the  deck  in  compartment  3- 

90-2-L  at  FR  96,  starboard  side  is  being  plugged/patched. 


NAVEDTRA  43119- G  Topic  Task 

3 1 8.2.9  Coordinate  Repair  Party  efforts  to  Say  steps  of  procedure 

combat/control  fire. 


Question.  list  the  steps  the  Repair  Party  Leader  would  take  in  fighting  a  fire  in  his/her 
area  of  responsibility. 

Answer.  (Reference  NSTM  555-5.3.2) 

1.  _ Determine  the  location  of  the  fire. 

2.  _ Determine  the  class  of  the  fire. 

3.  _ Determine  the  hazards  the  fire  poses  to  adjacent  spaces. 

4.  Order  fire  boundaries. 
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5.  _ Order  electrical  isolation. 

6.  _ Order  mechanical  isolation. 

7.  _ Order  smoke  boundaries. 

8.  Attack  the  fire. 


NAVEDTRA  43119- G  Topic  Task 

3 1 8.2. 1 3  Coordinate  Repair  Party  efforts  to  Say  reasons  for  step 

combat/control  structural  damage. 

Question.  List  the  reports  or  indications  the  Repair  Party  Leader  might  get  that  would 
cause  him  to  believe  shoring  is  required. 

Answer.  (Reference  NSTM  079V2,  079-44. 125) 

1.  _ Deep  bulges  in  plating. 

2.  _ Bowed/bent  frames. 

3.  _ Bowed/bent  stanchions. 

4.  _ Cracked  seams. 

5.  _ Panting  bulkhead. 


Question.  The  Scene  Leader  is  responsible  for  deciding  whether  to  have  the  firefighters 
wear  the  FireFighter’s  Ensemble.  You  have  a  new  scene  leader  who  asks  you  whether  his 
teams  should  wear  the  FFE.  What  factors  should  you  consider  in  your  response? 

Answer.  (Reference  NSTM  555-5.2. 1) 

1 .  _ Reduced  endurance  due  to  heat  stress. 

2.  _ Size  of  the  fire  (i.e.  are  the  FFE’s  necessary  for  a  small  fire?). 

3.  _ Progress  of  the  fire  while  the  fireparty  is  dressing  out 

(i.e.  is  the  fire  likely  to  spread  if  the  fireparty  is  delayed?) 


NAVEDTRA  43119- G  Topic  Task 

3 1 8.2. 1 3  Coordinate  Repair  Party  efforts  to  Say  reasons  for  step 

combat/control  structural  damage. 

Question.  Why  is  a  panting  bulkhead  a  dangerous  condition? 

Answer.  (Reference  NSTM  079V2,  079-44. 125) 

_ Panting  is  a  dangerous  condition  as  it  causes  metal  fatigue  which  in  time 

will  result  in  cracking  or  splitting. 


Question.  What  does  this  message  tell  you  as  Repair  Party  Leader?  (Show  Message 
Blank  Nr  31 
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Answer.  (Reference  NWP  3-20.31,  Figure  4.3,  sheet  7) 

_ Message  Nr  3  says  Progressive  flooding  reported  in  compartment  2-406- 

1-Q,  4”  deep 


NAVEDTRA  43119-  G  Topic  Task 

3 1 8.2.9  Coordinate  Repair  Party  efforts  to  Say  steps  of  procedure 

combat/control  fire. 

Question.  What  are  your  methods  of  communication  from  the  Repair  Locker  to  other 
stations? 

Answer.  (Reference  NWP3-20.3 1,  Section  4.1) 

1 .  _ Sound-powered  phones 

2.  _ Ship’s  Service  Telephones 

3.  _ Intercom  (ie.  4MC) 

4.  _ DC  WIFCOM 

5.  _ Messengers 


Question.  Your  scene  leader  calls  back  to  the  Locker  and  reports  that  he  needs  3  OBA 
relief  personnel.  You  look  around  the  Repair  Locker  area  and  there  are  no  extra 
personnel  available.  What  action  do  you  take  to  help  out  the  scene  leader? 

Answer.  _ Call  DC  Central  and  ask  the  DCA  to  find  and  send  additional 

personnel. 


Question.  What  does  this  message  tell  you  as  Repair  Party  Leader?  (Show  Message 
Blank  Nr  41 

Answer.  (Reference  NWP  3-20.31,  Figure  4.3,  sheet  1) 

_ Message  Nr  4  says  fire  boundaries  have  been  set,  primary  forward  FR 

200,  primary  aft  FR  245,  secondary  forward  FR175,  secondary  aft  FR  270. 


Question.  When  might  stability  of  the  ship  be  considered  a  problem? 

Answer.  (Reference  NWP  3-20.31,  Art  5.3.5) 

1 .  _ Small  or  negative  metacentric  height  with  a  logy,  long  period  roll 

2.  _ The  extent  of  flooding  exceeds  floodable  length 

3.  _ Excessive  list 

4.  _ Heavy  winds  and  seas  combined  with  existing  flooding 
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APPENDIX D.  DATASETS 


This  appendix  contains  the  data  sets  used  for  this  study.  To  simplify  the  analysis, 
integer  indicators  were  assigned  to  some  of  the  variables.  Table  9  lists  the  variable  and 
the  applicable  integer  indicator  employed.  In  order  to  incorporate  education  and  rank 


Variable 

Integer  Indicator 

Class  Convening  Date  (Mar  17, 1997) 

1 

Class  Convening  Date  (April  14,  1997) 

2. 

Class  Convening  Date  (May  12, 1997) 

3 

Control  Group 

0 

Treatment  Group 

1 

Male 

1 

Female 

0 

Enlisted  Surface  Warfare  (ESWS)  Qualified 

1 

Not  ESWS  Qualified 

0 

Table  9.  Assignment  of  Integer  Indicators. 

into  the  analysis,  integers  indicators  had  to  be  developed  for  those  two  variables.  In  the 
case  of  education,  the  DMDC  database  already  had  values  associated  with  particular 
levels  of  education  (Table  10).  Table  11  provides  the  indicators  assigned  to  the  rank/rate 
variable. 


Education  Variable 

Integer  Indicator 

3-4  Years  High  School  (No  diploma) 

5 

High  School  Diploma 

6 

2  Years  of  College 

8 

College  Graduate 

10 

Masters  or  Equivalent 

11 

High  School  GED 

13 

Table  10.  Education  Variable  Integer  Indicators. 


In  the  data  set,  E-5  represents  second  class  petty  officers.  These  subjects  can  be 
identified  as  being  those  whose  rate  designator  ends  with  “2”  (e.g.  AT2).  First  class  petty 
officers  are  those  whose  rate  designator  ends  with  “1”  (e.g.  HT1).  E-7  represents  a  Chief 
Petty  Officer  (e.g.  SKC),  while  W-2  corresponds  with  warrant  officers  (e.g.  CW02). 
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Among  commissioned  officers,  0-1  and  0-2  represent  Ensign  (ENS)  and  Lieutenant 
(junior  grade)  (LTJG),  while  0-3  and  above  corresponds  with  Lieutenants  and  senior. 


Rank/Rate  Variable 

Integer  Indicator 

E-5 

1 

E-6 

2 

E-7  through  W-4 

3 

0-1  through  0-2 

4 

0-3  and  above 

5 

Table  1 1.  Rank/Rate  Variable  Integer  Indicators 


Table  12  provides  the  complete  data  covering  all  three  classes  that  participated  in 
the  study  between  March  and  May  1997.  The  variable  “Sea  Time”  refers  to  the  amount 


Class 

Date 

Subject 

Nr 

Group 

Rank/ 

Rate 

Education  ESWS 

Gender 

SeaTime 

(mo) 

PreTest 

ExpTime 

(min) 

OBSl 

3/17/97 

1 

Treatment 

AT2 

6 

N 

F 

19 

66.67 

91 

38.78 

3/17/97 

2 

Treatment 

AOl 

6 

Y 

M 

64 

70.00 

6 

44.90 

3/17/97 

3 

Treatment 

DC2 

6 

Y 

M 

31 

76.67 

6 

44.90 

3/17/97 

4 

Treatment 

SMC 

6 

Y 

M 

120 

80.00 

6 

67.35 

3/17/97 

5 

Treatment 

ENC 

6 

Y 

M 

94 

46.67 

7 

59.18 

3/17/97 

6 

Treatment 

ENS 

10 

N 

M 

7 

73.33 

36 

53.06 

3/17/97 

7 

Treatment 

LT 

10 

N 

M 

43 

73.33 

3 

65.31 

3/17/97 

8 

Treatment 

ENS 

10 

N 

M 

12 

70.00 

22 

57.14 

3/17/97 

9 

Treatment 

ENS 

10 

N 

F 

8 

73.33 

32 

40.82 

3/17/97 

10 

Treatment 

ENS 

10 

N 

F 

0 

70.00 

34 

59.18 

3/17/97 

11 

Treatment 

CW03 

10 

N 

M 

52 

50.00 

179 

55.10 

3/17/97 

12 

Treatment 

LTJG 

10 

N 

M 

37 

80.00 

17 

28.57 

3/17/97 

13 

Control 

DM1 

6 

N 

M 

50 

23.33 

0 

22.45 

3/17/97 

14 

Control 

LCDR 

11 

N 

F 

1 

60.00 

0 

46.94 

3/17/97 

15 

Control 

RMC 

6 

N 

M 

75 

46.67 

0 

61.22 

3/17/97 

16 

Control 

CW03 

6 

Y 

M 

173 

76.67 

0 

61.22 

3/17/97 

17 

Control 

BM2 

6 

Y 

M 

120 

63.33 

0 

22.45 

3/17/97 

18 

Control 

SKC 

8 

Y 

F 

51 

46.67 

0 

40.82 

3/17/97 

19 

Control 

MMC 

6 

Y 

M 

111 

66.67 

0 

59.18 

3/17/97 

20 

Control 

HT1 

6 

N 

M 

63 

80.00 

0 

63.27 

3/17/97 

21 

Control 

LTJG 

10 

N 

M 

8 

73.33 

0 

38.78 

3/17/97 

22 

Control 

ENS 

10 

N 

M 

6 

63.33 

0 

48.98 

3/17/97 

23 

Control 

LTJG 

10 

N 

M 

17 

70.00 

0 

44.90 

4/14/97 

24 

Treatment 

GSM2 

6 

Y 

M 

52 

90.00 

46 

57.14 

4/14/97 

25 

Treatment 

EMI 

6 

N 

M 

97 

73.33 

22 

48.98 

4/14/97 

26 

Treatment 

HT1 

6 

N 

M 

75 

70.00 

20 

53.06 

4/14/97 

27 

Treatment 

GSE2 

6 

N 

M 

61 

60.00 

42 

51.02 
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Class  Subject  Group  Rank/  Education  ESWS  Gender  SeaTime  PreTest  ExpTime  OBS1 

Date  Nr  Rate  (mo)  (min) 


of  sea  time  the  subject  had  accumulated  (in  months)  up  to  the  date  the  Repair  Party 
Leader  School  class  commenced.  The  PreTest  column  provides  the  score  earned  by  the 
subject  on  the  pretest  administered  the  first  day  of  the  course.  Exposure  time  is  the 
amount  of  time  (in  minutes)  the  subject  spent  actually  working  scenarios  on  the  IDCTT 
(RPL)  computer.  This  was  calculated  by  taking  the  difference  between  scenario  start 
time  and  finish  time  for  each  scenario  run  by  the  members  of  the  treatment  group.  Start 
and  finish  times  are  automatically  recorded  by  the  system  and  are  stored  on  the 
computer's  hard  drive.  The  column  labeled  “OBS1”  provides  the  score  the  subject  earned 
on  the  post-treatment  oral  board  conducted  during  the  second  week  of  the  course. 

Table  13  provides  data  used  for  analysis  of  the  May  97  class  alone. 


Class 

Date 

Subject 

Nr 

Group 

Rank/ 

Rate 

Education  ESWS 

Gender 

SeaTime 

(mo) 

PreTest  ExpTime 
(min) 

OBS1 

5/12/97 

48 

Treatment 

ENS 

10 

0 

1 

0 

60.00 

12 

79.59 

5/12/97 

49 

Treatment 

DC2 

6 

1 

1 

29 

70.00 

24 

93.88 

5/12/97 

50 

Treatment 

DC1 

13 

1 

1 

38 

76.67 

22 

93.88 

5/12/97 

51 

Treatment 

PNC 

6 

1 

1 

60 

70.00 

48 

77.55 

5/12/97 

52 

Treatment 

ENS 

10 

0 

1 

0 

60.00 

26 

81.63 

5/12/97 

53 

Treatment 

LTJG 

10 

0 

1 

25 

76.67 

12 

69.39 

5/12/97 

54 

Treatment 

ENS 

10 

0 

1 

2 

66.67 

14 

71.43 

5/12/97 

55 

Treatment 

GSM1 

6 

1 

1 

102 

70.00 

48 

93.88 

5/12/97 

56 

Treatment 

SHI 

6 

1 

1 

77 

56.67 

12 

75.51 

5/12/97 

57 

Treatment 

LTJG 

10 

0 

1 

24 

76.67 

26 

75.51 

5/12/97 

58 

Treatment 

ENS 

10 

0 

0 

0 

66.67 

12 

69.39 

5/12/97 

59 

Treatment 

LT 

10 

0 

0 

31 

60.00 

10 

77.55 

5/12/97 

60 

Treatment 

LT 

10 

0 

1 

18 

43.33 

20 

81.63 

5/12/97 

61 

Control 

EN2 

6 

0 

1 

44 

60.00 

0 

38.78 

5/12/97 

62 

Control 

LTJG 

10 

0 

1 

11 

66.67 

0 

51.02 

5/12/97 

63 

Control 

ENS 

10 

0 

1 

6 

86.67 

0 

65.31 

5/12/97 

64 

Control 

DC1 

6 

1 

1 

97 

73.33 

0 

34.69 

5/12/97 

65 

Control 

LTJG 

10 

0 

1 

25 

70.00 

0 

48.98 

5/12/97 

66 

Control 

DC1 

13 

1 

1 

101 

63.33 

0 

40.82 

5/12/97 

67 

Control 

LTJG 

10 

0 

1 

25 

80.00 

0 

55.10 

5/12/97 

68 

Control 

LTJG 

10 

0 

1 

25 

73.33 

0 

36.73 

5/12/97 

69 

Control 

SKC 

6 

0 

1 

34 

63.33 

0 

38.78 

5/12/97 

70 

Control 

HT1 

6 

0 

0 

36 

60.00 

0 

42.86 

5/12/97 

71 

Control 

BMC 

6 

1 

1 

96 

40.00 

0 

32.65 

5/12/97 

72 

Control 

LTJG 

10 

0 

1 

26 

73.33 

0 

44.90 

5/12/97 

73 

Control 

ENS 

10 

0 

1 

10 

70.00 

0 

55.10 

Table  13.  May  1997  Data  Set 
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Table  14  provides  the  censored  data  set.  As  mentioned  in  Chapter  IV.B.l,  additional 
information  was  available  for  52  of  the  73  students  participating  in  the  study.  The 
censored  data  set  provided  as  Table  14  indicates  the  information  available  for  these 
subjects.  The  data  is  listed  by  subject  number;  all  information  listed  in  Table  12  is  the 
same.  In  the  race  category,  1  signifies  white,  2  is  black  and  3  is  other.  The  AFQT  column 


Subject  Nr 

Race 

AFQT 

Mental 

Cat 

SAR 

SPC 

SCS 

CAI 

1 

1 

93 

1 

64 

53 

53 

56.67 

2 

2 

42 

3.5 

47 

61 

51 

53.00 

3 

3 

51 

3 

47 

44 

52 

47.67 

4 

1 

62 

3 

58 

52 

45 

51.67 

5 

2 

59 

3 

61 

56 

38 

51.67 

6 

1 

98 

1 

64 

61 

65 

63.33 

7 

1 

69 

2 

55 

59 

57 

57.00 

10 

3 

84 

2 

62 

47 

71 

60.00 

11 

i 

58 

3 

61 

57 

38 

52.00 

12 

3 

66 

2 

58 

59 

46 

54.33 

13 

2 

33 

3.5 

57 

38 

43 

46.00 

14 

1 

78 

2 

62 

55 

58 

58.33 

16 

1 

67 

2 

54 

37 

43 

44.67 

18 

2 

51 

3 

52 

51 

70 

57.67 

19 

1 

78 

2 

61 

56 

57 

58.00 

20 

1 

39 

3.5 

56 

51 

41 

49.33 

21 

3 

94 

1 

59 

62 

53 

58.00 

24 

1 

81 

2 

62 

59 

46 

55.67 

25 

1 

75 

2 

62 

59 

44 

55.00 

26 

2 

50 

3 

52 

51 

68 

57.00 

27 

1 

44 

3.5 

51 

53 

49 

51.00 

28 

1 

59 

3 

53 

51 

45 

49.67 

29 

3 

64 

3 

61 

53 

60 

58.00 

30 

1 

75 

2 

62 

60 

50 

57.33 

31 

1 

33 

3.5 

45 

50 

44 

46.33 

35 

2 

47 

3.5 

57 

49 

61 

55.67 

36 

1 

56 

3 

55 

56 

42 

51.00 

37 

3 

17 

4.33 

50 

42 

42 

44.67 

39 

1 

83 

2 

63 

61 

46 

56.67 

40 

1 

87 

2 

64 

55 

61 

60.00 

41 

1 

94 

1 

65 

54 

69 

62.67 

42 

1 

78 

2 

59 

56 

58 

57.67 

43 

1 

54 

3 

56 

55 

42 

51.00 

44 

1 

45 

3.5 

48 

50 

49 

49.00 

45 

1 

63 

3 

57 

54 

46 

52.33 

46 

1 

98 

1 

51 

63 

64 

59.33 

47 

1 

86 

2 

66 

64 

45 

58.33 

49 

1 

84 

2 

61 

62 

68 

63.67 
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Subject  Nr 

Race 

AFQT 

Mental  Cat 

SAR 

SPC 

SCS 

CAI 

50 

1 

71 

2 

58 

59 

57 

58.00 

51 

i 

54 

3 

56 

55 

42 

51.00 

55 

1 

64 

3 

54 

58 

53 

55.00 

56 

3 

44 

3.5 

56 

50 

54 

53.33 

57 

1 

94 

1 

63 

61 

70 

64.67 

61 

3 

36 

3.5 

47 

42 

46 

45.00 

63 

1 

92 

2 

62 

54 

56 

57.33 

64 

1 

74 

2 

54 

54 

65 

57.67 

65 

1 

79 

2 

59 

51 

65 

58.33 

66 

2 

35 

3.5 

51 

50 

45 

48.67 

67 

1 

96 

1 

64 

56 

53 

57.67 

69 

2 

67 

2 

54 

56 

48 

52.67 

70 

1 

76 

2 

64 

56 

48 

56.00 

71 

1 

41 

3.5 

69 

45 

54 

56.00 

Table  14.  Censored  Data  Set 


provides  the  percentile  score  achieved  by  the  subject  on  the  Armed  Forces  Qualification 
Test  taken  at  the  time  of  entry  into  the  service.  The  Mental  Category  assigned  to  the 
individual  is  based  on  the  AFQT  percentile,  as  indicated  in  Table  15.  SAR,  SPC,  SCS 
and  CAI  are  as  described  in  Chapter  IV.B.2.  The  “S”  indicates  that  the  scores  used  were 
standardized  by  DMDC  to  account  for  variations  between  examinations. 


AFQT  Score 

Mental  Category 

Integer  Indicator 

1  to  9 

V 

5 

10  to  15 

IV(c) 

4.66 

16  to  20 

IV(b) 

4.33 

21  to  30 

IV(a) 

4 

31  to  49 

ffl(b) 

3.5 

50  to  64 

m(a) 

3 

65  to  92 

n 

2 

93  to  99 

i 

1 

Table  15.  AFQT  Scores  and  Associated  Mental  Categories. 
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APPENDIX  E.  STEPWISE  REGRESSION  RESULTS 


Stepwise  regression,  was  the  primary  tool  used  to  aid  in  the  selection  of  variables 
for  this  model.  Further  information  regarding  this  technique  is  available  in  various 
books,  including  the  reference  listed  in  Chapter  V.A.  S-Plus  was  used  to  conduct  the 
stepwise  regression,  and  the  results  of  this  process  is  provided  in  this  appendix.  Table  16 
provides  the  results  for  the  complete  data  set,  while  Tables  17  and  18  refer  to  the  May 
1997  and  Censored  data  sets,  respectively. 


4^87*^ 

...  . 

3X 

lrss:  _ 1 

16480.2 

14053.89  13784.41 

13516.54 

12878.84 

12842.31 

12811.78  12803.99 

12803.33| 

size: 

— 

1 

2 

3 

4 

5 

6 

7 

8 

_ ^ 

which: 

Class 

Group 

Rank 

Education 

ESWS 

Gender 

SeaTime 

PreTest 

ExpTime 

l(+2) 

F 

T 

F 

F 

F 

F 

F 

F 

F 

2(+l) 

T 

T 

F 

F 

F 

F 

F 

F 

F 

3(+8) 

T 

T 

F 

F 

F 

F 

F 

T 

F 

4(+3) 

T 

T 

T 

F 

F 

F 

F 

T 

F 

5(+5) 

T 

T 

T 

F 

T 

F 

F 

T 

F 

6(+4) 

T 

T 

T 

T 

T 

F 

F 

T 

F 

7(+9) 

T 

T 

T 

T 

T 

F 

F 

T 

T 

8(+7) 

T 

T 

T 

T 

T 

F 

T 

T 

T 

9(+6) 

T 

T 

T 

T 

T 

T 

T 

T 

T 

f.stat: 

11.91957 

12.08505  1.348916 

1.347617 

3.317538 

0.187727 

0.154911  0.038944 

0. 00321 7| 

Table  16.  Stepwise  Regression  Results  (Complete  data  set). 
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^  r  7V“?£-  V,  -  '  ~  ?' 

rss 

1  2029.515  1672.756  1529.69  1410.136  1174.59  1134.164  1122.389  1122.327 

size _ 

1  2  3  4  5  6  7  8 


which 


Class 

Group 

Rank 

Education 

ESWS 

Gender 

SeaTime 

PreTest 

ExpTime 

l(+2) 

F 

T 

F 

F 

F 

F 

F 

f" 

F 

2(+8) 

F 

T 

F 

F 

F 

F 

F 

T 

F 

3(+9) 

F 

T 

F 

F 

F 

F 

F 

T 

T 

4(+7) 

F 

T 

F 

F 

F 

F 

T 

T 

T 

5(+5) 

F 

T 

F 

F 

T 

F 

T 

T 

T 

6(+4) 

F 

T 

F 

T 

T 

F 

T 

T 

T 

7(+3) 

F 

T 

T 

T 

T 

F 

T 

T 

T 

8(+6) 

F 

T 

T 

T 

T 

T 

T 

T 

T 

f.stat 

94.20182  4.905348 

2.057581 

1.780417 

4.010695 

0.677225 

0.188848  0.000935] 

Table  17.  Stepwise  Regression  Results  (May  1997  data  set). 
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APPENDIX  F.  RESULTS  OF  AN  OVA 


Analysis  of  variance  (ANOVA)  was  used  to  refine  the  model  suggested  by 
stepwise  regression.  Table  19  is  the  output  provided  by  S-Plus  for  the  complete  data  set. 
As  stated  in  Chapter  V.A.3,  the  stepwise  regression  identified  five  variables  to  be 


Df 

Sum  of  Sq 

Mean  Sq 

F  Value 

Pr(F) 

Group 

1 

2766.718 

2766.718 

23.62394 

0.0000176 

Class 

1 

2426.314 

2426.314 

20.71736 

0.0000468 

PreTest 

1 

269.479 

269.479 

2.30097 

0.1369662 

Rank 

1 

267.87 

267.87 

2.28723 

0.1381127 

ESWS 

1 

637.702 

637.702 

5.44509 

0.024607 

Group:Class 

1 

3067.522 

3067.522 

26.19239 

0.0000077 

Group:  PreTest 

1 

901.118 

901.118 

7.6943 

0.008301 

Class:  PreTest 

1 

29.886 

29.886 

0.25518 

0.6161537 

Group:  Rank 

1 

39.352 

39.352 

0.33601 

0.565311 

Class:Rank 

1 

252.927 

252.927 

2.15964 

0.1493099 

PreTest:  Rank 

1 

143.189 

143.189 

1.22263 

0.2752905 

Group:  ESWS 

1 

226.284 

226.284 

1.93215 

0.1720248 

Class:  ESWS 

1 

1.019 

1.019 

0.0087 

0.9261202 

PreTest:  ESWS 

1 

103.144 

103.144 

0.88071 

0.3535037 

Rank:  ESWS 

1 

69.546 

69.546 

0.59383 

0.4453605 

Group:Class:PreT  est 

1 

5.656 

5.656 

0.04829 

0.8271532 

Group:Class:Rank 

1 

355.76 

355.76 

3.0377 

0.0888477 

Group:  PreT  est:  Rank 

1 

37.679 

37.679 

0.32172 

0.5736663" 

Class:  PreT  est:  Rank 

1 

35.669 

35.669 

0.30456 

0.5840326 

Group:Class:ESWS 

1 

0.651 

0.651 

0.00556 

0.9409365 

Group:  PreT  est:ESWS 

1 

1.356 

1.356 

0.01158 

0.9148351 

Class:PreT  est:ESWS 

1 

18.757 

18.757 

0.16016 

0.6910846 

Group:  Rank:  ESWS 

1 

1408.606 

1408.606 

12.02754 

0.0012454 

Class:  Rank:  ESWS 

1 

151.548 

151.548 

1.29401 

0.2619146 

PreT  est:Rank:ESWS 

1 

235.005 

235.005 

2.00662 

0.1641676 

Group:  Class:  PreT  est:Rank 

1 

570.478 

570.478 

4.87109 

0.0329664 

Group:Class:PreT  est:ESWS 

1 

0.015 

0.015 

0.00013 

0.9910162 

Group:Class:Rank:ESWS 

1 

180.19 

180.19 

1.53857 

0.2218821 

Group:PreT  est:Rank:ESWS 

1 

3.152 

3.152 

0.02691 

0.8704972 

Class:PreT  est:Rank:ESWS 

1 

238.532 

238.532 

2.03673 

0.161111 

Group:Class:PreT  est:Rank:ESWS 

1 

0.084 

0.084 

0.00071 

0.9788252 

Residuals 

41 

4801.716 

117.115 

Table  19.  Analysis  of  Variance  (Complete  data  set). 
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considered  for  inclusion  in  the  model.  This  ANOVA  was  based  on  those  variables  and 
considered  all  interactions. 


Table  20  provides  the  output  for  ANOVA  of  the  May  1997  data  set.  Stepwise 
regression  identified  six  variables  in  this  case.  Again,  interactions  were  considered  by 
the  model. 


■ 

isr^ 

gill 

I 

Df 

Sum  of  Sq 

Mean  Sq 

F  Value 

Pr(F) 

Group 

1 

7966 

7966 

180.7974 

0.000889 

PreTest 

1 

356.759 

356.759 

8.097 

0.065349 

ExpTime 

1 

143.066 

143.066 

3.2471 

0.169348 

SeaTime 

1 

119.554 

119.554 

2.7134 

0.198064 

ESWS 

1 

235.546 

235.546 

5.346 

0.103828 

Education 

1 

40.426 

40.426 

0.9175 

0.408795 

Group:  PreTest 

1 

182.365 

182.365 

4.139 

0.134765 

PreTest:  ExpTime 

1 

25.097 

25.097 

0.5696 

0.505256 

Group:SeaTime 

1 

248.67 

248.67 

5.6438 

0.098 

PreTest:SeaTime 

1 

2.804 

2.804 

0.0636 

0.817117 

ExpTime:SeaTime 

1 

29.16 

29.16 

0.6618 

0.475486 

Group:  ESWS 

1 

0.012 

0.012 

0.0003 

0.987719 

PreTest:  ESWS 

1 

106.72 

106.72 

2.4221 

0.217492 

ExpTime:  ESWS 

1 

125.069 

125.069 

2.8386 

0.190614 

SeaTime:ESWS 

1 

64.645 

64.645 

1.4672 

0.312501 

Group:Education 

1 

16.942 

16.942 

0.3845 

0.579115 

PreT  est:  Education 

1 

28.688 

28.688 

0.6511 

0.478758 

SeaTime:Education 

1 

38.076 

38.076 

0.8642 

0.421143 

ESWS:Education 

1 

9.022 

9.022 

0.2048 

0.681617 

Group.PreT  est:SeaTime 

1 

79.741 

79.741 

1.8098 

0.271182 

PreT  est:  ExpTime:  SeaTime 

1 

41.012 

41.012 

0.9308 

0.405822 

Group:  PreT  est:  ESWS 

1 

3.96 

3.96 

0.0899 

0.783916 

Residuals 

3 

132.181 

44.06 

Table  20.  Analysis  of  Variance  (May  1997  data  set). 

Table  21  provides  the  results  of  ANOVA  for  the  censored  data  set.  Stepwise 
regression  identified  six  variables,  and  these  were  further  examined  by  ANOVA. 


-  ...  -  r 

Df  SumofSq 

Mean  Sq  F  Value 

Pr(F) 

Race 

1  1848.419 

1848.419  22.67897 

0.0010264 

Group 

1  1833.695 

1833.695  22.4983 

0.0010541 

108 


ESWS 

1 

1024.092 

1024.092 

12.56498 

0.0062676 

Class 

1 

597.253 

597.253 

7.32792 

0.0241151 

Pre.TestScore 

1 

476.693 

476.693 

5.84873 

0.0387111 

Rank 

1 

227.722 

227.722 

2.79401 

0.128951 

Race:  Group 

1 

108.059 

108.059 

1.32581 

0.2792213 

Race:  ESWS 

1 

131.198 

131.198 

1.60972 

0.2363613 

Group:ESWS 

1 

829.158 

829.158 

10.17326 

0.0110144 

Race:Class 

1 

213.854 

213.854 

2.62385 

0.1397207 

Group:Class 

1 

1253.432 

1253.432 

15.37884 

0.0035025 

ESWS:  Class 

1 

12.227 

12.227 

0.15002 

0.7075132 

Race:Pre.T  estScore 

1 

163.938 

163.938 

2.01142 

0.1898031 

Group:Pre.T  estScore 

1 

825.495 

825.495 

10.12831 

0.0111399 

ESWSrPre.T  estScore 

1 

22.057 

22.057 

0.27063 

0.6154652 

Class:Pre.T  estScore 

1 

53.86 

53.86 

0.66083 

0.437242 

Race:Rank 

1 

570.646 

570.646 

7.00147 

0.0266491 

Group:  Rank 

1 

4.797 

4.797 

0.05885 

0.8137546 

ESWS:  Rank 

1 

250.14 

250.14 

3.06906 

0.1137081 

Class:  Rank 

1 

10.245 

10.245 

0.1257 

0.7310961 

Pre.T  estScore:Rank 

1 

46.447 

46.447 

0.56988 

0.4696002 

Race:Group:ESWS 

1 

452.659 

452.659 

5.55384 

0.0428352 

Race:Group:Class 

1 

1.258 

1.258 

0.01543 

0.9038705 

Race:ESWS:Class 

1 

4.092 

4.092 

0.05021 

0.8277026 

Group:ESWS:Class 

1 

208.327 

208.327 

2.55605 

0.1443372 

Race:Group:Pre.T  estScore 

1 

315.862 

315.862 

3.87543 

0.0805217 

Race:ESWS:Pre.T  estScore 

1 

35.505 

35.505 

0.43563 

0.5257699 

Group:  ESWS  Pre.T  estScore 

1 

18.538 

18.538 

0.22745 

0.644788 

Race:Class:Pre.T  estScore 

1 

179.472 

179.472 

2.20201 

0.1719847 

Group:Class:Pre.T  estScore 

1 

15.524 

15.524 

0.19047 

0.6728119 

ESWS:Class:Pre.T  estScore 

1 

12.792 

12.792 

0.15695 

0.7012067 

Race:Group:Rank 

1 

79.774 

79.774 

0.97878 

0.34835 

Race:ESWS:Rank 

i 

109.023 

109.023 

1.33764 

0.2772258 

Group:ESWS:Rank 

i 

455.537 

455.537 

5.58916 

0.0423133 

Race:Class:Rank 

i 

61.968 

61.968 

0.76032 

0.40589 

Group:Class:Rank 

i 

201.166 

201.166 

2.46818 

0.1506208 

ESWS:  Class:  Rank 

i 

0.262 

0.262 

0.00322 

0.9560002 

Race:Pre.T  estScore:Rank 

i 

79.74 

79.74 

0.97836 

0.3484488 

Group:Pre.T  estScore:Rank 

i 

147.054 

147.054 

1.80426 

0.2120857 

ESWS:  Pre.T  estScore:  Rank 

i 

208.93 

208.93 

2.56345 

0.143824 

Class:Pre.T  estScore:Rank 

i 

9.872 

9.872 

0.12112 

0.7358237 

Group:Class:Pre.T  estScore:Rank 

i 

8.991 

8.991 

0.11031 

0.7473905 

Residuals 

9 

733.533 

81.504 

Table  21.  Analysis  of  Variance  (Censored  data  set). 


APPENDIX  G.  POWER  FUNCTION  GENERATION 

This  appendix  provides  the  S-Plus  function  that  was  used  to  develop  the  power 
function  curves  shown  in  Chapter  V.  As  shown,  this  function  requires  six  input  values 
and  provides  a  graphical  output. 


>  FindPower 

function(mean. delta,  x,  y,  data.l,  data.2,  LvlofSig) 

{ 

#  mean,  delta  is  the  vector  of  mean  values  for  which  the  user  wants  to  calculate  the  power 
of  the  test  (i.e.  mean.deha_seq(-20,20,0.01) 

#  x  is  sample  variance  of  treatment  group 

#  y  is  sample  variance  of  control  group 

#  data.l  is  treatment  group  data  set  dependent  variable,  a  vector 

#  data.2  is  control  group  data  set  dependent  variable,  a  vector 

#  LvlofSig  is  the  desired  level  of  statistical  significance 

alpha  <-  1  -  LvlofSig 
for(i  in  l:length(mean.  delta))  { 
if(mean.delta[i]  >  0)  { 

z[i]  <-  qnorm(alpha)  -  (mean.deha[i])/(x/length(data.l)  +  y/ 
length(data.2))A0.5 

} 

else  if(mean.delta[i]  <  0)  { 

z[i]  <-  qnorm(alpha)  +  (mean.deha[i])/(x/length(data.l)  +  y/ 
length(data.2))A0. 5 

} 

else  if(mean.delta[i]  =  0)  { 

z[i]  <-  qnorm(alpha)  -  (mean.deha[i])/(x/length(data.l)  +  y/ 
length(data.2))/'0.5 

} 

t  <-  pnorm(z) 

} 

win.graph() 

plot(mean. delta,  1  - 1,  type  =  "1",  main  =  "Power  Function  for  Data",  ylab  = 
"Power",  xlab  =  "Difference  between  Data  Means") 
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